After over 600 hours of work, I finally digitized and organized them properly so the cassettes can be thrown away.
Part 2
This is what the footage looks like now:
All family videos have been digitized and are available for viewing from a private media server.
513 separate video clips turned out. Each has a title, description, date of recording, tags for all participants, indicating the age at the time of recording. Everything rests on a private media server that only family members have access to, and hosting costs less than $ 1 a month.
This article explains everything I have done, why it took eight years - and how to achieve the same result much easier and faster.
First naive attempt
Around 2010, my mom bought some sort of VHS to DVD converter and ran all of our home videos through it.
The original DVDs that Mom made (I don’t know what happened to the missing letters)
The problem is that Mom only made one set of DVDs. All relatives live in different states, so it was inconvenient to transfer discs in a circle.
In 2012, my sister gave me these DVDs. I copied the video files and put everything in the cloud storage. Problem solved!
Family Video DVD Rips on Google Cloud
A few weeks later, I asked if anyone had watched the tapes. It turned out that no one was looking. Even I didn't look. In the age of YouTube, it's stupid to download three-hour files of unknown content in search of interesting footage.
Only my mother was delighted: "Great," she said, "now can we finally throw out all these tapes?"
Oh-oh. This is a terrible question. What if we missed some entries? What if cassettes can be digitized at higher quality? What if the labels contain important information?
I have always found it uncomfortable to throw originals away until I am absolutely certain that the video was copied at the highest possible quality. Thus, I had to get down to business myself.
I had no idea what I was getting myself into.
Doesn't sound so hard
If you don’t understand why it took me eight years and hundreds of hours, I don’t blame you. I also thought that everything would be easy.
This is how the digitization process looks from start to finish:
More precisely, this is how it looks in theory. Here's how it happened in practice:
Most of the time was spent reworking what has already been done. I finished one stage, and then after one or two stages I found some kind of flaw in the technique. I had to go back and redo. For example, I shot a video of 20 tapes before realizing that the audio was slightly out of sync. Or, after several weeks of editing, I discovered that I was exporting a video in a format that does not support streaming on the Internet.
To keep the reader's sanity, I present the process as if he systematically moved forward, so as not to force you to constantly jump back and redo everything, as I had to.
Step 1. Capture video
Okay, back to 2012. Mom really wanted to throw away the cassettes that she had kept for twenty years, so when we first met, she immediately handed me a huge cardboard box. So my digitization quest began.
The obvious solution was to outsource the work to professionals. There are many digitization companies, and some specialize in home video.
But I am quite sensitive to privacy and I did not want strangers to view our family video with intimate moments of personal life, including my potty training (at the appropriate age; nothing strange!). And I also thought there was nothing difficult about digitizing.
Spoiler alert: It turned out to be really difficult.
First attempt to capture video
My father still had an old family VCR, so I asked him to dig it out of the basement for the next family dinner. I bought a cheap RCA to USB adapter from Amazon and got down to business.
TOTMC video capture device , the first of many A / V devices I bought during my long quest
To process video from a USB capture device I used VirtualDub, the 2012 version is a little outdated but not critical.
VirtualDub footage of me reading a book to my father at the age of four
Attack with distorted sound
When I started the editing process, I noticed a slight out of sync between audio and video. Okay, no problem. I can move the sound a little.
Ten minutes later, it was out of sync again. Didn't I move it a little the first time?
It gradually dawned on me that audio and video are not just out of sync, they are actually recorded at different speeds. Throughout the entire tape, they diverge more and more. To synchronize, I had to manually adjust the sound every few minutes.
If your rig captures audio and video at different rates, the only solution is to manually correct the audio every few minutes.
Can you imagine how difficult it is to distinguish between audio 10 milliseconds earlier or 10 milliseconds later? It's really hard! Judge for yourself.
In this video, I'm playing with my poor, patient kitten called Black Magic. The sound is slightly out of sync. Determine if he is ahead of the picture or is late?
An example of a video clip with out-of-sync sound and picture
At this point Black Magic jumps, a fragment with a fivefold slowdown:
Out of sync sound and picture, slowing down by five times
Answer : the sound comes with a delay of a few milliseconds.
Maybe spend an extra hundred dollars instead of hundreds of hours of personal time?
Sound correction alone took many hours of tedious, maddening work. In the end, it occurred to me that it was possible to avoid out-of-sync by using a better and more expensive video capture device. After doing some research, I bought a new one on Amazon:
My second attempt at acquiring a video capture device
Even with a new device, the out of sync has not disappeared anywhere.
VCR with "super" prefix
Maybe the problem is with the VCR. On the digitizing forums , it was said that there would be no desynchronization on a VCR with a "time-based corrector" (TBC), this feature is available on all Super VHS (S-VHS) VCRs.
Well, of course! Why would I mess around with a stupid regular VCR when there is a super- VCR that solves the problem?
No one makes S-VHS VCRs anymore, but they are still available on eBay. For $ 179, I bought a JVC SR-V10U model that seems to work well for digitizing VHS:
Vintage JVC SR-V10U VCR I bought on eBay for $ 179
The "Super" VCR came by mail. After months of struggling with out-of-sync audio, I was overjoyed to have a piece of equipment that would solve all my problems.
I opened the box, plugged everything in - but the sound was still recorded at a different speed. Eh.
Tedious troubleshooting and years of struggle
I have embarked on a pathetic attempt at troubleshooting. It was painful to watch. Each time I pulled all the equipment out of the closet, crawled on my knees behind the desktop to connect everything, tried to capture a video - and again I watched that nothing happened.
Here I came across a random post on a forum from 2008, which talks about installing some strange Chinese driver without a signature ... This is a terrible idea, but I'm desperate. However, he did not help.
I have tried different digitizing programs. Bought a special VHS cassette to clean the VCR magnetic heads. I bought a third video capture device . Nothing helped.
I invariably gave up, turned everything off and hid the equipment in the closet for several more months.
We give up and give the cassettes to the professionals
2018 has come. I was dragging videotapes and tons of equipment to four different apartments and was about to move from New York to Massachusetts. I could not find the strength to carry them again, because I already realized that I would never finish this project on my own.
I asked the family if they could donate the tapes to a digitization firm. Fortunately, no one objected - everyone wanted to see the tapes again.
Me : But that means some company will have access to all of our home videos. Does it suit you?
Sister : I don't care. It worries you alone. Wait, so you could have just paid someone from the start?
Me : Uh-uh ...
Digitizing all 45 cassettes costs $ 750. It seems expensive, but by that time I would have paid whatever I want, just not to deal with this equipment anymore.
When they gave the files, the video quality was definitely better. In my frames, distortions were always visible at the edges of the frame, but the specialists digitized everything without any distortions. Most importantly, audio and video are perfectly synchronized.
Here's a video comparing professional digitization and my homebrew attempts:
Comparison of professional and homemade digitizing in the video where my mom films my first attempt at programming
Step 2. Editing
In home filming, about 90% of the material is boring, 8% is interesting, and 2% is amazing. After digitizing, you still have a lot of work to do.
Editing in Adobe Premiere
On a VHS cassette, a long stream of video clips is interspersed with blank sections. To edit the tape, you must define where each clip begins and ends.
For editing, I used Adobe Premiere Elements, which costs less than $ 100 for a lifetime license. Its most important feature is a scalable timeline. It allows you to quickly find the boundaries of a scene, and then zoom in to find the exact video frame where the clip begins or ends.
Critical timeline with scaling in Adobe Premiere Elements
The problem with Premiere is that the process requires a lot of manual steps, but it takes a long time to digitize and export. Here is my sequence of operations:
- Open a raw file that contains 30-120 minutes of video.
- Mark the boundaries of an individual clip.
- Export clip.
- Wait 2-15 minutes for the export to complete.
- Repeat steps 2-4 until the tape runs out.
The long wait meant that I was constantly switching between video editing and some other task, shifting my attention back and forth for several hours.
Another disadvantage was non-reproducibility. Fixing a small bug was almost as difficult as doing it from scratch. It hit me hard when it came to posting videos. It was only then that I realized that in order to stream on the Internet, it was necessary to initially export the video to a format that web browsers natively support. I was faced with a choice: restart the tedious process of exporting hundreds of clips, or re-encode the exported videos to a different format with degraded quality.
Editing automation
After a lot of time spent on manual work, I wondered if there was any way to apply AI here. Determining the boundaries of clips seems to be a suitable task for machine learning. I knew the accuracy would not be perfect, but let him do at least 80% of the work and I'll fix the last 20%.
I experimented with a tool called pyscenedetect , which analyzes video files and gives out timestamps where scene changes take place:
$ docker run \
--volume "/videos:/opt" \
handflucht/pyscenedetect \
--input /opt/test.mp4 \
--output /opt \
detect-content --threshold 80 \
list-scenes
[PySceneDetect] Output directory set:
/opt
[PySceneDetect] Loaded 1 video, framerate: 29.97 FPS, resolution: 720 x 480
[PySceneDetect] Downscale factor set to 3, effective resolution: 240 x 160
[PySceneDetect] Scene list CSV file name format:
$VIDEO_NAME-Scenes.csv
[PySceneDetect] Detecting scenes...
[PySceneDetect] Processed 55135 frames in 117.6 seconds (average 468.96 FPS).
[PySceneDetect] Detected 33 scenes, average shot length 55.7 seconds.
[PySceneDetect] Writing scene list to CSV file:
/opt/test-Scenes.csv
[PySceneDetect] Scene List:
-----------------------------------------------------------------------
| Scene # | Start Frame | Start Time | End Frame | End Time |
-----------------------------------------------------------------------
| 1 | 0 | 00:00:00.000 | 1011 | 00:00:33.734 |
| 2 | 1011 | 00:00:33.734 | 1292 | 00:00:43.110 |
| 3 | 1292 | 00:00:43.110 | 1878 | 00:01:02.663 |
| 4 | 1878 | 00:01:02.663 | 2027 | 00:01:07.634 |
...The instrument did indeed show an accuracy of about 80%, but it took longer to validate than it saved. However, pyscenedetect made one of the most important discoveries for the entire project: scene boundary definition and clip export are separate tasks.
I remembered that I am a programmer
Up to this point, everything I did in Adobe Premiere was considered “editing”. Cutting out clips from raw frames seemed inextricably linked to finding the bounds of a clip because that is how Premiere presented the task. When pyscenedetect printed out the metadata table, it made me realize that I could separate the scene search from the video export. It was a breakthrough.
The reason editing was so tedious and time consuming was because I had to wait for Premiere to export each clip. If I wrote the metadata into a spreadsheet and wrote a script that automatically exports the video, the editing process would fly by.
Moreover, spreadsheets have greatly expanded the amount of metadata. Initially, I crammed metadata into the file name, but this limits them. Having a whole spreadsheet allowed me to catalog a lot more information about the clip, such as who is in it, when it was recorded, and any other data I want to show during the video demo. A giant spreadsheet with metadata about my home videos I was later able to use this metadata to add information to clips, such as how old we were and a detailed description of what was going on in the clip. Spreadsheet functionality allows you to record metadata that provides more information about clips and makes them easier to view
The success of the automated solution
With the spreadsheets in hand, I wrote a script that sliced the raw video into clips based on the CSV data.
Here's a footage of what it looks like in action:
At this point, I've spent hundreds of hours tediously selecting the boundaries of a clip in Premiere, pressing export, waiting a few minutes for it to finish, and then starting over. Moreover, the process was repeated several times on the same clips, when quality problems were found later.
As soon as I automated part of the clipping, a huge load fell off my shoulders. I no longer had to worry about forgetting metadata or choosing the wrong output format. If you get an error later, you can just correct the script and repeat everything.
Part 2
Digitizing and editing videos is only half the battle. We still need to find a convenient option for publishing on the Internet, so that all relatives watch the family video in a convenient format with streaming like on YouTube.
In the second part of this article, I will go into detail on how to get an open source media server with all the video clips running, which costs me only 77 cents a month.
Continuation,
Part 2
