New Project - "Uncle Bob"

Where all the user feedback and suggestions for improvements go
Post Reply
big_wheeler
Posts: 1
Joined: Wed Jan 12, 2011 3:19 am

New Project - "Uncle Bob"

Post by big_wheeler »

Hello,

I have been using comskip on sage for a while now and it works great.

On Sage I also have all of my home movies. I had the idea on how great it would be to show "Uncle Bob" scenes from my home movies that he was in.

The concept seems very similar to comskip but instead knowing where to skip the commercials in the video "Uncle Bob" would know which video and at what spot in the video to start playing.

Identifying the People or Events in the movies would be a manual process but once done an html type page could list all the People and Events and you would be able to click on the link and the player would open the video to the scene you selected.

It seems like this feature would go beyond your "Uncle Bob". You could identify your favorite scenes in movies as well. Who wouldn't want to jump right to the "Horse getting shot" scene in Animal house.

If any smart people have any idea on how to create this thing and package it the same way comskip is that would be great.

Thanks from me and "Uncle Bob"

Reavers
Posts: 4
Joined: Mon Mar 07, 2011 8:29 am

Re: New Project - "Uncle Bob"

Post by Reavers »

This should be possible with some work.

You'd need a program that would seperate movie files into jpegs with timestamps in the title or as metadata. I'm sure they're out there. To cut down the work it would need to do it maybe every 15 seconds. That would be 480 photos for a 2 hr movie.

Picasa, from Google, has photo face recognition which lets you tag faces/photos. Picasa will group sets of photos of the same person so it shouldn't take too long. It will sometimes not be sure about photos so those might take a little time. I haven't messed with it too much. The faster your computer the better for this step.

Next step(big step) would be to write a simple app to convert this meta data/time stamps into comskip files. It would take some time but actually shouldn't be too hard.
Picasa stores their meta data in their own file system. You'd have to have your app figure it out or there's also an app you can download that can add picasa tags directly to the jpeg itself as metadata. Then you could just parse the jpegs themselves with your app and convert the data to comskip files. http://www.anvo-it.de/wiki/avpicfacexmptagger:main
There also configuration on how long there hasn't been a photo with the tag Uncle Bob in it till you start determining where to skip and such but that shouldn't be too hard either. (I say that but I know thing always go wrong :) )

Can anyone think why this wouldn't work?

erik
Site Admin
Posts: 3338
Joined: Sun Aug 21, 2005 3:49 pm

Re: New Project - "Uncle Bob"

Post by erik »

This is a great idea.
The piece missing is how to tell picasa to tag and group unknown faces as you don't want to have to name the faces yourself.
Not sure I will spend some time implementing it but if anyone is able to provide me with a text file with one line per 15 seconds sample telling with numbers which of the overall identified faces are present on this sample I will implement in comksip the reading of this file as additional input.

Reavers
Posts: 4
Joined: Mon Mar 07, 2011 8:29 am

Re: New Project - "Uncle Bob"

Post by Reavers »

I may attempt this. It would be a fun project. I'd probably have to do it in C# though. No promises! :)

The Piscasa step would be all manual labor. You do have to manual tag the face groups it recognizes. It's actually a good idea to tag the unknown faces as well since they could be recognized by the user as Uncle Bob. Picasa is pretty good at guessing unknowns and grouping them. I'll have to do some testing. If you didn't want to tag unknowns then I guess you wouldn't have to. You'd just have a less accurate comskip file.

Reavers
Posts: 4
Joined: Mon Mar 07, 2011 8:29 am

Re: New Project - "Uncle Bob"

Post by Reavers »

It seems I can do most of it with ffmpeg. Still have to determine more of it's abilities but it looks this way. Should be able to test it in a simulated run soon.

Reavers
Posts: 4
Joined: Mon Mar 07, 2011 8:29 am

Re: New Project - "Uncle Bob"

Post by Reavers »

I'll probably use Python instead of C# for cross platform builds. It'll be open source of course, cause it has to be :wink:. The Linux version of Picasa doesn't have the face recognition in it yet but hopefully it will in the future.

**
n/m :roll: , Didn't realize edl files were that easy to make :D
**

Looks like Picasa's face info is pretty easy to get to in the database. So for this app the only external apps you would need would be Picasa and ffmpeg.

jobormo
Posts: 3
Joined: Tue Mar 08, 2011 2:54 pm

Re: New Project - "Uncle Bob"

Post by jobormo »

it is possible to integrate comskip to an application developed in c#¿?

erik
Site Admin
Posts: 3338
Joined: Sun Aug 21, 2005 3:49 pm

Re: New Project - "Uncle Bob"

Post by erik »

Comskip can be started in a hidden window.
Read the output to stdout and stderr to see the progress.
It is difficult to otherwise mix the C and C# code.

Post Reply