This presentation explains how to use YOLOv5 to store video only when a specific object is detected. It will also explain how to store video in the AWS cloud.
Purpose
When viewing surveillance video later, it is possible to reduce the time and effort required to search for surveillance targets and reduce the storage capacity if the video that does not include the target of surveillance can be automatically deleted. This article describes a system that uses YOLOv5 to automatically delete video that does not include the monitored object and store only the video that does include the monitored object in the cloud. This time, AWS S3 will be used as the cloud storage.
Plan
I would like to implement the following plan.
- YOLOv5 performs object detection and saves the video locally every minute.
- If a monitored object is included in the video locally saved, it is uploaded in the cloud.
- If a monitored object is not included in the video locally saved, it is deleted.
Implementation
This time, we will implement it using YOLOv5 with the python version of OpenCV used in this article.
The source code is available in this repository.
The main function is explained below.
def main(is_cuda, video, class_list, object_ids, s3_bucket):
executor = ThreadPoolExecutor()
colors = [(255, 255, 0), (0, 255, 0), (0, 255, 255), (255, 0, 0)]
net = build_model(is_cuda)
capture = cv2.VideoCapture(video)
start = time.time_ns()
frame_count = 0
total_frames = 0
fps = -1
is_save = False
writer = None
timestamp = datetime.now().strftime("%Y%m%d%H%M")
video_name = f"{timestamp}.mp4"
while True:
_, frame = capture.read()
if frame is None:
print("End of stream")
break
inputImage = format_yolov5(frame)
outs = detect(inputImage, net)
class_ids, confidences, boxes = wrap_detection(inputImage, outs[0])
if len([c for c in class_ids if c in object_ids]) != 0:
is_save = True
frame_count += 1
total_frames += 1
for (classid, confidence, box) in zip(class_ids, confidences, boxes):
color = colors[int(classid) % len(colors)]
cv2.rectangle(frame, box, color, 2)
cv2.rectangle(
frame, (box[0], box[1] - 20), (box[0] + box[2], box[1]), color, -1
)
cv2.putText(
frame,
class_list[classid],
(box[0], box[1] - 10),
cv2.FONT_HERSHEY_SIMPLEX,
0.5,
(0, 0, 0),
)
if frame_count >= 30:
end = time.time_ns()
fps = 1000000000 * frame_count / (end - start)
frame_count = 0
start = time.time_ns()
if fps > 0:
fps_label = "FPS: %.2f" % fps
cv2.putText(
frame, fps_label, (10, 25), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2
)
cv2.imshow("output", frame)
if writer is None and fps > 0:
w, h = frame.shape[1], frame.shape[0]
writer = cv2.VideoWriter(
video_name, cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h)
)
if writer:
writer.write(frame)
# update timestamp
new_timestamp = datetime.now().strftime("%Y%m%d%H%M")
if timestamp != new_timestamp:
writer.release()
writer = None
if is_save:
print(f"save {video_name}")
executor.submit(upload, s3_bucket, video_name)
else:
print(f"remove {video_name}")
os.remove(video_name)
is_save = False
timestamp = new_timestamp
if cv2.waitKey(1) & 0xFF == ord("q"):
print("finished by user")
running = False
if writer:
writer.release()
break
Code language: Python (python)build_model returns an object of the YOLOv5 model. If is_cuda is True and CUDA is available, it returns an object using CUDA, otherwise, it returns an object using CPU.
def build_model(is_cuda):
net = cv2.dnn.readNet("config_files/yolov5s.onnx")
if is_cuda:
print("Attempty to use CUDA")
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA_FP16)
else:
print("Running on CPU")
net.setPreferableBackend(cv2.dnn.DNN_BACKEND_OPENCV)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU)
return netCode language: Python (python)cv2.VideoCapture opens the device with the web camera device number specified by video. 0 is specified if the web camera device is /dev/video0.
capture = cv2.VideoCapture(video)Code language: Python (python)Get video frames with capture.read()
_, frame = capture.read()Code language: Python (python)The frame is converted with format_yolov5, the object is detected with detect, and the detected object’s ID (class_id), the confidence level of the detected object (confidences), and the bounding boxes (boxes) are converted with wrap_detection.
inputImage = format_yolov5(frame)
outs = detect(inputImage, net)
class_ids, confidences, boxes = wrap_detection(inputImage, outs[0])Code language: Python (python)If the ID (class_id) of the detected object is included in the object_ids passed as the main argument, is_save is set to True in order to save the video being created. If object_ids is set to a person’s ID (=0), is_save will be set to True and the video will be saved when that ID (=0) is detected. See config_files/classes.txt for the ID numbers.
if len([c for c in class_ids if c in object_ids]) != 0:
is_save = TrueCode language: Python (python)cv2.VideoWriter_fourcc specifies the format of the video to be saved. “mp4v” is specified as the format. The available video formats depend on the OS, supported codecs, and OpenCV’s ffmpeg support. This article may be helpful. fps (Frame Per Second) specifies the most recent fps.
writer = cv2.VideoWriter(
video_name, cv2.VideoWriter_fourcc(*"mp4v"), fps, (w, h)
)Code language: Python (python)The timestamp is updated every minute in the format YYYYmmddHHMM.
# update timestamp
new_timestamp = datetime.now().strftime("%Y%m%d%H%M")
if timestamp != new_timestamp:
writer.release()
writer = NoneCode language: Python (python)If is_save is True, the upload thread is invoked with executor.submit and the locally stored video (video_name) is uploaded to the cloud. If False, the locally stored video is simply deleted.
if is_save:
print(f"save {video_name}")
executor.submit(upload, s3_bucket, video_name)
else:
print(f"remove {video_name}")
os.remove(video_name)Code language: Python (python)In upload, the file is uploaded to S3 by calling Bucket.upload_file using the AWS API python library boto3. The uploaded file is no longer needed, so it is deleted with os.remove.
def upload(s3_bucket, file):
if s3_bucket:
s3 = boto3.resource("s3")
bucket = s3.Bucket(s3_bucket)
# upload
bucket.upload_file(file, file)
print(f"upload {file} done.")
# remove
os.remove(file)Code language: Python (python)AWS
The following operations require an AWS account.
In addition, administrator privileges are required to run the AWS CLI.
To upload files to AWS S3, prepare an S3 bucket and a dedicated IAM User that can upload files to the S3 bucket. The following is an AWS CloudFormation template.
AWSTemplateFormatVersion: 2010-09-09
Description: Create IAM Users
Parameters:
UserName:
Type: String
Description: username to upload files to s3
BucketName:
Type: String
Description: bucket name to create
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
S3UploadUser:
Type: AWS::IAM::User
Properties:
UserName: !Ref UserName
Policies:
- PolicyName: S3UploadOnly
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- s3:PutObject
Resource:
- !Sub arn:aws:s3:::${BucketName}/*
DependsOn: S3BucketCode language: YAML (yaml)The permission required for boto3’s Bucket.upload_file is s3:PutObject, so only grant that permission. The S3 bucket name is global, so if you specify a bucket name that already exists, the bucket creation will fail. DependsOn: S3Bucket is specified so that S3UploadUser is created only when the bucket is created successfully.
Run the following to create the S3 bucket yolov5-opencv-save-detected and the IAM user my-s3-uploader.
aws cloudformation deploy --template-file template.yaml --stack-name my-stack-name --parameter-overrides UserName=my-s3-uploader BucketName=yolov5-opencv-save-detected --capabilities CAPABILITY_NAMED_IAMCode language: Bash (bash)After creating the IAM user my-s3-uploader above, use the following command to obtain an access key.
$ aws iam create-access-key --user-name my-s3-uploader
{
"AccessKey": {
"UserName": "my-s3-uploader",
"AccessKeyId": "xxxxxxxxxxxxxxxxxxxx",
"Status": "Active",
"SecretAccessKey": "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"CreateDate": "2022-10-13T12:41:00+00:00"
}
}Code language: Bash (bash)Specify the obtained AccessKeyId and SecretAccessKey as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, respectively, and execute the program as follows. Specify the bucket name as –s3-bucket.
Run
AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxx
AWS_SECRET_ACCESS_KEY=yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
python3 python/yolo.py --s3-bucket yolov5-opencv-save-detectedCode language: Bash (bash)Confirm that the video in which the person is detected is uploaded to the S3 bucket yolov5-opencv-save-detected.
$ aws s3 ls s3://yolov5-opencv-save-detected/
2022-10-13 22:10:01 908341 202210132209.mp4
2022-10-13 22:14:01 666566 202210132213.mp4Code language: Bash (bash)All codes can found at https://github.com/otamajakusi/yolov5-opencv-save-detected
That’s all.
