Build 81.012 Problem

Here you can ask your questions on how to use Comskip for the detection of commercials. Also questions on how to remove commercials are welcome
Post Reply
AlphaZulu
Posts: 39
Joined: Wed Feb 02, 2011 3:33 am

Build 81.012 Problem

Post by AlphaZulu »

With the above build, Comskip has screeched to a snails pace. V80.042 Utilized about 16-20% of processor consumption (AMD 6 Core). V81.012 Utilizes about 2% for about 10 seconds and then 0% for a few min. A 30 min show takes about 1.5 hrs to scan.

I had to revert back to version 80.042 (which has a problem with my MPEG-PS recordings)

Recording Specs: (Media Info.) After VRD Quick Stream Fix

Code: Select all

Format                           : MPEG-TS
File size                        : 3.51 GiB
Duration                         : 33mn 58s
Overall bit rate mode            : Variable
Overall bit rate                 : 14.8 Mbps

Video
ID                               : 4096 (0x1000)
Menu ID                          : 1 (0x1)
Format                           : MPEG Video
Format version                   : Version 2
Format profile                   : Main@High
Format settings, BVOP            : Yes
Format settings, Matrix          : Custom
Codec ID                         : 2
Duration                         : 33mn 58s
Bit rate mode                    : Variable
Bit rate                         : 13.7 Mbps
Maximum bit rate                 : 90.0 Mbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 59.940 fps
Standard                         : Component
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Compression mode                 : Lossy
Bits/(Pixel*Frame)               : 0.248
Stream size                      : 3.25 GiB (92%)

Audio
ID                               : 4097 (0x1001)
Menu ID                          : 1 (0x1)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : 129
Duration                         : 33mn 58s
Bit rate mode                    : Constant
Bit rate                         : 384 Kbps
Channel(s)                       : 6 channels
Channel positions                : Front: L C R, Side: L R, LFE
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Stream size                      : 93.3 MiB (3%)
Language                         : English

Text #1
ID                               : 4096 (0x1000)-608-1
Menu ID                          : 1 (0x1)
Format                           : EIA-608
Muxing mode                      : A/53 / DTVCC Transport
Muxing mode, more info           : Muxed in Video #1
Duration                         : 33mn 58s
Bit rate mode                    : Constant
Stream size                      : 0.00 Byte (0%)

Text #2
ID                               : 4096 (0x1000)-1
Menu ID                          : 1 (0x1)
Format                           : EIA-708
Muxing mode                      : A/53 / DTVCC Transport
Muxing mode, more info           : Muxed in Video #1
Duration                         : 33mn 58s
Bit rate mode                    : Constant
Stream size                      : 0.00 Byte (0%)
Here are the specs of the original Recording

Code: Select all

Format                           : MPEG-PS
File size                        : 3.49 GiB
Duration                         : 33mn 58s
Overall bit rate mode            : Variable
Overall bit rate                 : 14.7 Mbps

Video
ID                               : 224 (0xE0)
Format                           : MPEG Video
Format version                   : Version 2
Format profile                   : Main@High
Format settings, BVOP            : Yes
Format settings, Matrix          : Custom
Duration                         : 33mn 58s
Bit rate mode                    : Variable
Maximum bit rate                 : 90.0 Mbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate                       : 59.940 fps
Standard                         : Component
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Compression mode                 : Lossy

Audio #1
ID                               : 128 (0x80)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Compression mode                 : Lossy

Audio #2
ID                               : 189 (0xBD)-128 (0x80)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Muxing mode                      : DVD-Video
Duration                         : 33mn 52s
Bit rate mode                    : Constant
Bit rate                         : 384 Kbps
Channel(s)                       : 6 channels
Channel positions                : Front: L C R, Side: L R, LFE
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Delay relative to video          : 4s 144ms
Stream size                      : 93.1 MiB (3%)

Text #1
ID                               : 224 (0xE0)-608-1
Format                           : EIA-608
Muxing mode                      : A/53 / DTVCC Transport
Muxing mode, more info           : Muxed in Video #1
Bit rate mode                    : Constant
Stream size                      : 0.00 Byte (0%)

Text #2
ID                               : 224 (0xE0)-1
Format                           : EIA-708
Muxing mode                      : A/53 / DTVCC Transport
Muxing mode, more info           : Muxed in Video #1
Bit rate mode                    : Constant
Stream size                      : 0.00 Byte (0%)
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Build 81.012 Problem

Post by erik »

Does the slowdown happen when running multiple instances of comskip or also when running one single instance?
AlphaZulu
Posts: 39
Joined: Wed Feb 02, 2011 3:33 am

Re: Build 81.012 Problem

Post by AlphaZulu »

erik wrote:Does the slowdown happen when running multiple instances of comskip or also when running one single instance?
A single instance.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Build 81.012 Problem

Post by erik »

Will test on all my systems as I have never observed this till today.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Build 81.012 Problem

Post by erik »

First test results on a Athlon X3 400e using the test recording you uploaded.
Running three instances of comskip 0.81.0123 on a HD 1080i MPEGstream.
Top speed on all 3 instances around 110fps
Average speed around 80fps
Speed sometimes for some seconds drops to below 20fsp with very low CPU usage due to disk activities by demigrator.exe (I'm running on a window home server)

Are you sure there are no other high priority disk intensive applications active?
AlphaZulu
Posts: 39
Joined: Wed Feb 02, 2011 3:33 am

Re: Build 81.012 Problem

Post by AlphaZulu »

erik wrote:First test results on a Athlon X3 400e using the test recording you uploaded.
Running three instances of comskip 0.81.0123 on a HD 1080i MPEGstream.
Top speed on all 3 instances around 110fps
Average speed around 80fps
Speed sometimes for some seconds drops to below 20fsp with very low CPU usage due to disk activities by demigrator.exe (I'm running on a window home server)

Are you sure there are no other high priority disk intensive applications active?
Not sure if it matters, but I'm running Win7 64bit. Using dpclat.exe, with 4 HD tuners hitting the Raid set, I run about 250us max, (my system is not in danger until it his around 1400us) so there seems to be plenty of IO bandwidth to the drives. Total CPU utilization is about 12%.

EDIT:
The problem seems to be intermittent, as I just tried a recording (30 min long) that I ended after 2.35 hrs into a scan, and this time it completed as it should with no problems.
nate
Posts: 22
Joined: Wed Oct 19, 2011 5:37 am

Re: Build 81.012 Problem

Post by nate »

I've also noticed some circumstances where comskip runs really slow, though I'm using 0.80.042 so it mightn't be the same problem. I'm running Win7 x64 on a Dual core E8400.

Running comskip with nothing else running I get 400-500fps with around 50% cpu usage (which is essentially fully utilising 1 core), however I noticed that when I have an mencoder process running in the background that comskip runs at only 30-60fps with around 1% cpu. The other 99% being taken by mencoder.

The first thing I thought was that it was disk contention, but I tried with the mencoder process using files on a different physical drive than the file comskip is processing and it still slows down comskip, so it's not a disk limitation.

I run the mencoder process with it's priority set to Idle and noticed that comskip also starts with it's process priority set to Idle, so I tried manually increasing comskip's priority in Process Explorer to High but it makes no difference. mencoder was still using 99% cpu.

At this point I decided to remove mencoder from the equation and see if it was simply cpu usage that was the issue, so i whipped up a very simple c# console app which set the process priority to Idle and then maxes out the cpus.

Code: Select all

static void Main(string[] args)
{
	System.Diagnostics.Process.GetCurrentProcess().PriorityClass = System.Diagnostics.ProcessPriorityClass.Idle;
	while (true)
	{
		// Using Parallel.For to fully utilise all cpus (requires .NET 4)
		System.Threading.Tasks.Parallel.For(0, 10000000, (i) => { int j = i * i; /* do some math to use cpu */ });
	}
}
If I run comskip while this program is running then it also runs slow, even with comskip's priority raised to High.
When I set the affinity of my app to not use one of the cpus then comskip comes back up to full speed using that cpu.

So from what I've seen, it seems that comskip is always giving precedence to other programs even if comskip has the higher process priority.

Is this perhaps something to do with the play_nice implementation? Mine are currently set to
play_nice_start=-1
play_nice_end=-1
play_nice_sleep=0

cheers,
Nate
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Build 81.012 Problem

Post by erik »

I removed all speed reduction except playnice options in build 0.81.015
Could you test?
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Build 81.012 Problem

Post by erik »

Make the 0.81.016 as 015 has a bug
AlphaZulu
Posts: 39
Joined: Wed Feb 02, 2011 3:33 am

Re: Build 81.012 Problem

Post by AlphaZulu »

erik wrote:I removed all speed reduction except playnice options in build 0.81.015
Could you test?
I think that did it, seems like it's working as it should. Will have a busy recording day tomorrow, so that should be a good testing day..


Thanks for all your Help Eric.
nate
Posts: 22
Joined: Wed Oct 19, 2011 5:37 am

Re: Build 81.012 Problem

Post by nate »

0.81.016 is working great. Comskip now keeps running at full speed when another Idle process using the rest of the CPU.

Nice work erik.
Post Reply