Comskip Support Forum

Comskip is a free commercial detector, browse the forum for more information
It is currently Thu Sep 20, 2018 1:03 am

All times are UTC [ DST ]




Post new topic Reply to topic  [ 13 posts ] 
Author Message
PostPosted: Mon Jan 22, 2018 10:19 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
Hi, I'm trying to use Comskip compiled from the latest git in a Debian 9.3 x86_64 VM.
I built Comskip in this way:
Code:
apt-get install -y git build-essential libargtable2-dev libsdl1.2-dev libavformat-dev autoconf libtool
git clone git://github.com/erikkaashoek/Comskip
cd Comskip
./autogen.sh
./configure
make
and it appears to run successfully, but when I pass a cutscene dump file (created from the Windows comskip binary) it crashes.
I'm unable to capture the output directly because it prints all the libraries like below (and output redirection doesn't help):

Image

So I've tried running the same command under strace and the output is available at this gist:

Code:
https://gist.github.com/virtualdj/bc09f4073769924b52761e76e71ecede#file-gistfile1-txt-L916-L928

It crashes after reading the cutscene file "./paramount1.dmp" (which is in the same directory than "comskip" and "paramount.ini"), on line 928.

The strange thing is that if I run the same command line under "valgrind" instead of "strace" it doesn't crash and it creates the output correctly! Is this a bug?


Top
 Profile  
Reply with quote  
PostPosted: Sat Jan 27, 2018 3:58 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
Is there anything I can do to provide more information to solve this issue?
I really would like to run Comskip in Linux and then move it into my NAS (which has the same crashing issues with cutscene, this is why I'm trying debugging in a clean Debian VM).

EDIT:
Even by setting "verbose=10" on the INI file, the content of the log is a laconic:
Code:
Loaded 24325 bytes from cutfile "paramount1.dmp"
and nothing else. Doesn't seem a permission issue, as I'm running Comskip as root.


Top
 Profile  
Reply with quote  
PostPosted: Wed Feb 07, 2018 7:31 am 
Offline
Site Admin

Joined: Sun Aug 21, 2005 3:49 pm
Posts: 3209
Thanks, I see there is a warning on memory corruption or a double free.
Helps to find the problem.


Top
 Profile  
Reply with quote  
PostPosted: Wed Feb 07, 2018 8:18 am 
Offline
Site Admin

Joined: Sun Aug 21, 2005 3:49 pm
Posts: 3209
What is the size of the generated cutscene dump file?


Top
 Profile  
Reply with quote  
PostPosted: Wed Feb 07, 2018 7:09 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
erik wrote:
Thanks, I see there is a warning on memory corruption or a double free.
Helps to find the problem.
Thanks!
erik wrote:
What is the size of the generated cutscene dump file?
Actually I've generated the "paramount1.dmp" and "paramount2.dmp" on a different machine (using ComSkip GUI for Windows) and then I'm moving the files to a headless Debian VM which crashes. So basically I'm not "generating" anything on the VM that crashes.
And the same files work in the Windows version.

The size of each of the two dump files is exactly 24329 bytes.


Top
 Profile  
Reply with quote  
PostPosted: Wed Feb 07, 2018 9:44 pm 
Offline
Site Admin

Joined: Sun Aug 21, 2005 3:49 pm
Posts: 3209
Please test first with files generated on linux just to make sure the file format is the same
Does not have to be relevant dump file, just to test reading


Top
 Profile  
Reply with quote  
PostPosted: Thu Feb 08, 2018 7:25 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
OK, so I've created another subfolder test2 to test creating cutscenes directly on Linux VM.
I cannot use a GUI, so I hope the following is correct to "extract" cutscenes from the video file:
Code:
root@debian9:~/cskip/Comskip/test/test2# ../comskip --dump=944 --output=. ../Sample.ts
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --dump=944 --output=. ../Sample.ts

No INI file found in current directory.  Searching PATH...
INI file found at comskip.ini
No INI file found anywhere!!!!
Setting dump frame number to 944 as per command line.
 0:09:00 - 15498 frames in 24.79 sec(625.17 fps), 1.00 sec(632.00 fps), 99%
15534 frames decoded in 24.84 seconds (625.36 fps)
Commercials were found.

root@debian9:~/cskip/Comskip/test/test2# mv Sample.dmp sample1.dmp

root@debian9:~/cskip/Comskip/test/test2# ../comskip --dump=1359 --output=. ../Sample.ts
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --dump=1359 --output=. ../Sample.ts

No INI file found in current directory.  Searching PATH...
INI file found at comskip.ini
No INI file found anywhere!!!!
Setting dump frame number to 1359 as per command line.
 0:08:36 - 14893 frames in 22.80 sec(653.20 fps), 1.00 sec(622.00 fps), 95%
15534 frames decoded in 23.71 seconds (655.17 fps)
Commercials were found.

root@debian9:~/cskip/Comskip/test/test2# mv Sample.dmp sample2.dmp
root@debian9:~/cskip/Comskip/test/test2# rm *.txt
root@debian9:~/cskip/Comskip/test/test2# ls -l
total 48
-rw-r--r-- 1 root root 24329 Feb  8 19:13 sample1.dmp
-rw-r--r-- 1 root root 24329 Feb  8 19:14 sample2.dmp

root@debian9:~/cskip/Comskip/test/test2# md5sum sample*.dmp
b1928bec7009573ebebcbd46d31b4181  sample1.dmp
b1928bec7009573ebebcbd46d31b4181  sample2.dmp
Two questions here:
  1. Is it correct that comskip has to read the whole video just to extract a single frame? I deliberately supplied a missing INI file just to extract the cutscene without scanning, is this correct or is there a faster solution? Can I extract multiple frame numbers with one single command line?
  2. Why even if I wrote 2 different frame numbers the content of the two DMP files (actually the hash) is the same? The 2 dmp files that I extracted on Windows are actually different, though.

Then I copied my INI file and changed the filenames and, as you can see, it crashes.
Code:
root@debian9:~/cskip/Comskip/test/test2# cp ../paramount.ini sample.ini

< edit with nano >

root@debian9:~/cskip/Comskip/test/test2# grep "cutscenefile" sample.ini
cutscenefile1="./sample1.dmp"
cutscenefile2="./sample2.dmp"
cutscenefile3=""
cutscenefile4=""
cutscenefile5=""
cutscenefile6=""
cutscenefile7=""
cutscenefile8=""

root@debian9:~/cskip/Comskip/test/test2# ../comskip --ini=./sample.ini --output=. ../Sample.ts
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --ini=./sample.ini --output=. ../Sample.ts

Setting ini file to ./sample.ini as per commandline
Using ./sample.ini for initiation values.
Loaded 24325 bytes from cutfile "./sample1.dmp"
*** Error in `../comskip': double free or corruption (top): 0x000055b5038e88b0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bcb)[0x7fd42df5fbcb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76f96)[0x7fd42df65f96]
/lib/x86_64-linux-gnu/libc.so.6(+0x777de)[0x7fd42df667de]
/lib/x86_64-linux-gnu/libc.so.6(fclose+0x132)[0x7fd42df55c12]
../comskip(+0x12ae4)[0x55b4ff280ae4]
../comskip(+0x2c2d5)[0x55b4ff29a2d5]
../comskip(+0x3863)[0x55b4ff271863]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7fd42df0f2b1]
../comskip(+0x433a)[0x55b4ff27233a]
I hope this helps!


Top
 Profile  
Reply with quote  
PostPosted: Thu Feb 08, 2018 7:35 pm 
Offline
Site Admin

Joined: Sun Aug 21, 2005 3:49 pm
Posts: 3209
It helps to exclude certain errors.
Indeed you have to scan the whole file for a single cutscene to be created, this because seeking is not reliable on certain recordings


Top
 Profile  
Reply with quote  
PostPosted: Thu Feb 08, 2018 8:20 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
OK, that's fair. I hope I'll be able to move the DMP file from Windows to Linux, though.
If you need something else to track the crash, let me know.


Top
 Profile  
Reply with quote  
PostPosted: Thu Feb 08, 2018 9:34 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
I tried running under GDB and this is the result (unfortunately I'm not keen on C/C++):
Code:
root@debian9:~/cskip/Comskip/test/test2# gdb --args ../comskip --ini=./sample.ini --output=. ../Sample.ts
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
... cut ...
Reading symbols from ../comskip...done.

(gdb) run
.. cut ..
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff56b53fa in __GI_abort () at abort.c:89
#2  0x00007ffff56f1bd0 in __libc_message (do_abort=do_abort@entry=2,
    fmt=fmt@entry=0x7ffff57e6dd0 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff56f7f96 in malloc_printerr (action=3,
    str=0x7ffff57e6f08 "double free or corruption (top)", ptr=<optimized out>,
    ar_ptr=<optimized out>) at malloc.c:5049
#4  0x00007ffff56f87de in _int_free (av=0x7ffff5a1ab00 <main_arena>,
    p=0x5555580498d0, have_lock=0) at malloc.c:3905
#5  0x00007ffff56e7c12 in _IO_new_fclose (fp=0x5555580498e0) at iofclose.c:84
#6  0x0000555555563dfa in LoadCutScene (
    filename=filename@entry=0x555557f05780 <cutscenefile1> "sample1.dmp")
    at comskip.c:9961
#7  0x0000555555566ae4 in LoadIniFile () at comskip.c:8591
#8  0x00005555555802d5 in LoadSettings (argc=<optimized out>,
    argv=0x7fffffffebb8) at comskip.c:9068
#9  0x0000555555557863 in main (argc=4, argv=0x7fffffffebb8) at mpeg2dec.c:2174


Top
 Profile  
Reply with quote  
PostPosted: Fri Feb 09, 2018 8:24 am 
Offline
Site Admin

Joined: Sun Aug 21, 2005 3:49 pm
Posts: 3209
Many thanks for helping me.
I think I found and solved the problem.
The updated source is on github so if you can fetch the latest comskip version and rebuild and test?


Top
 Profile  
Reply with quote  
PostPosted: Fri Feb 09, 2018 8:07 pm 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
erik wrote:
Many thanks for helping me.
I think I found and solved the problem.
The updated source is on github so if you can fetch the latest comskip version and rebuild and test?

Thanks to you! It works perfectly now, also with the DMP files moved from Windows:
Code:
root@debian9:~/cskip/Comskip/test/test2# ../comskip --ini=./sample.ini --output=. ../Sample.ts
Comskip 0.81.098, made using ffmpeg
Donator build
The commandline used was:
../comskip --ini=./sample.ini --output=. ../Sample.ts

Setting ini file to ./sample.ini as per commandline
Using ./sample.ini for initiation values.
Loaded 24325 bytes from cutfile "sample1.dmp"
Loaded 24325 bytes from cutfile "sample2.dmp"
Mpeg:   ../Sample.ts
Exe     comskip.exe
Logo:   ./Sample.logo.txt
Ini:    ./sample.ini

Detection Methods to be used:
        1) CutScenes


ComSkip throttles back from -0001 to -0001.
The time is now 1833 so it's full speed ahead
... cut ...
----------------------------------------------------
Block list after weighing
----------------------------------------------------
  #     sbf  bs  be     fs     fe        ts        te       len     sc   scr cmb   ar                   cut    bri logo   vol sil   corr stdev   cc
  0:+     0   0   0      1  13559     0.00s   542.28s   542.28s   1.00  0.16   0 0.00  7      {       t    br 12780 0.00 10900 820  0.000   127           

13559 Frames Processed
Commercials were not found.


Top
 Profile  
Reply with quote  
PostPosted: Sat Feb 10, 2018 9:47 am 
Offline

Joined: Sat Jan 20, 2018 11:52 am
Posts: 15
Another small thing... I noticed when compiling the latest git version that it still displays version 0.81.098 when running the program, while you actually changed the version to 0.82.005 in comskip.h.
So I've proposed a PR in GitHub to automatically sync the version number.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 13 posts ] 

All times are UTC [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group