Multi-threading problem

[renzz]

I've been using the donator version (0.81.094) for a few days and am seeing a problem related I think to multi threading.

My PC is a quad-core but presents as 8 cores in task manager, so I have set number of threads to 8 in comskip.ini. If I start it running, it detects the logo, creates the logo file, and then starts to process the file at around 80 fps. In Task Manager, comskip.exe seems to only be using 13% cpu, ie just one core.

If I then ctrl-c the process and simply restart it without deleting the logo file, it detects it's there and then starts processing at over 280fps! CPU is around 80-90% so all cores are being used.

I've zipped up the logs from both runs as well as the ini file and eventual output files and they can be found at https://we.tl/AD7OAEO42J

Is this a bug in the way it multi-threads?

[erik]

Thank for the info
Will look into this as it seems to be a bug

[erik]

Did you try running both situations multiple time (e.g. deleting the logo file between run's) to ensure the hdd disk access is not making a difference?
The first time you run comskip on a new file the file will be read into memory. Next run will go much quicker as the disk is no longer the bottleneck
Will depend of course on file size and available memory

[renzz]

Did you try running both situations multiple time (e.g. deleting the logo file between run's) to ensure the hdd disk access is not making a difference?
The first time you run comskip on a new file the file will be read into memory. Next run will go much quicker as the disk is no longer the bottleneck
Will depend of course on file size and available memory

Yes, I have tried running both situations several times, and the problem is consistent. It doesn't seem like it's hammering the hdd, so I don't think it's that. Just watching the Comskip process in Task Manager, it looks like it uses 13 threads across all cpu cores while it's doing the logo detection and then drops down to five threads on one cpu core when it slows down. When running it with the logo file already there it uses 13 threads across all cores from the start and continues like that for the whole run.

I've tried this on both H264 and MPEG2 files and it's the same, but MPEG2 is much faster (as expected) running at around 450fps when it uses all cores.

[erik]

I'm currently not able to reproduce that behaviour as h.s64 never uses more then 2 cores (after logo is detected or when logo file is given) where MPEG2 always uses as many cores as I specify
So difficult to make progress for the moment

[renzz]

I'm currently not able to reproduce that behaviour as h.s64 never uses more then 2 cores (after logo is detected or when logo file is given) where MPEG2 always uses as many cores as I specify
So difficult to make progress for the moment

Ok, I'm happy to provide any more diags if you need them

[erik]

Seems its better now with build 0.81.095
Working on a transition to newer libavcodec which will have faster decoding and can use all cores on many formats.