A common use of the worker is to automatically catalog an entire volume or folder and keep the catalog up to date as new files are added. To do this efficiently and provide more flexibility and control over the process this is normally done by breaking it down into a number of related tasks:
- Use Quick Folder Sync to quickly scan the volume and create generic stub clips for each file, permitting files to be accessed or archived via CatDV as soon as they appear on disk. Set a clip status field to ‘Awaiting analysis’.
- Perform a server query to search for files of status ‘Awaiting analysis’ and perform media analysis on these to extract thumbnails and technical metadata. This will add each file to the task queue for separate processing but can run as a low priority background task. On completion, update the status to ‘Analysed’.
- Use another server query task to transcode the files and create proxy versions as required, either automatically (perhaps all video clips) or manually (based on the user selecting the clips they want proxies for within CatDV and ticking a checkbox to say ‘Create proxy’). On completion set the status to say ‘Proxy created’.
- Finally, you can have a bulk server query that tests whether all the files listed in the catalog are still available online and/or whether proxy files exist, and can update other clip fields to say ‘Original file exists’ and ‘Proxy file exists’. This provides the converse to quick folder sync and will identify clips that exist in the catalog but have since been deleted from disk.
- If desired, a separate task could be used to automatically delete both the clip from the catalog and the corresponing proxy file if an original file on the monitored folder is deleted. This isn’t recommended however. To avoid the disk of losing metadata deleting clips is usually better done manually (files may be offline only temporarily, if a folder is accidentally moved or a drive isn’t mounted for example, and you wouldn’t want to delete your clips in those circumstances).
These steps can be combined if required, for example combining ii) and iii) to create proxies at the same time as the files are analysed.
Unless the volume or folder being monitored is very small you may want to place clips into different catalogs based on the folder hierarchy. For example, if you have a watch folder on /Volumes/SAN and files are organised into a hiearchy based on customer name and project number then you might want to create a separate catalog for each project. The $P variable contains the path of each file relative to the watch folder root but as the project may contain subfolders we want to use the first two components of $P as the catalog name. We can do this using regular expressions, first to convert Windows paths to Unix-style paths if necessary then pull out the first two groups of characters between a forward slash. /[^/]+/ matches a / followed by one or more non-/ characters followed by another /. The Quick Folder Sync action might therefore publish to catalog name:
SAN$P{s,\,/,g}{s,(^/[^/]+/[^/]+)/.*,$1}
When checking whether clips are still online at step iv) we can search for clips where the catalog name starts with “SAN”.
(You might also want further statuses such as ‘Ready for logging’ and ‘Logged’ and move clips or files to another catalog or library location once they are logged, but this depends on your desired workflow.)