Disk IO - large disk read buffer layer

Where all the user feedback and suggestions for improvements go
Post Reply
K24A3
Posts: 23
Joined: Mon Feb 10, 2014 12:03 pm

Disk IO - large disk read buffer layer

Post by K24A3 »

Does comskip use a file read buffer where it reads the video from the disk in say 64MB chunks into RAM, or does the comskip processor read directly from disk?

It would be great if comskip had an extra file read buffer layer in between the comskip processor and the physical disk.

I implemented a read buffer in my own c# app and it made a huge difference in performance. I'm almost convinced a system like this would resolve all the IO hangs I'm experiencing.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Disk IO - large disk read buffer layer

Post by erik »

Comskip uses a big buffer when reading the recording to avoid an IO overload
But you need to be aware that modern CPU's are so fast that processing at full speed can consume the full bandwidth of a single disk or network drive
K24A3
Posts: 23
Joined: Mon Feb 10, 2014 12:03 pm

Re: Disk IO - large disk read buffer layer

Post by K24A3 »

I see. Can we add an ini option to specify the buffer size in the next release? Maybe it is too large for SSD's that read at 500MB/s with super fast access times ... way faster than any HDD which max out at about 150MB/s. Perhaps it needs to be re-tuned for todays SSD performance.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Disk IO - large disk read buffer layer

Post by erik »

Is there an actual problem?
Or is this a nice to have option?
K24A3
Posts: 23
Joined: Mon Feb 10, 2014 12:03 pm

Re: Disk IO - large disk read buffer layer

Post by K24A3 »

I have had ongoing system hangs. I have tried different INI settings but but nothing seems to work. The hangs happen less now but when they do I need to reset the PC to recover.

One configuration I didn't try was to start comskip at the start of the recording in MP, I always had it set to trigger at the end of the recording. Since then, there have been no IO hangs :) *fingers crossed*

I'll let you know if it continues to happen, if it does then maybe we can look into adding an option for the disk buffer size since I can't think of anything else that could resolve the issue.
Post Reply