Avs saving utf8

Here you can ask your questions on how to use Comskip for the detection of commercials. Also questions on how to remove commercials are welcome
Post Reply
Dimsok
Posts: 9
Joined: Fri Oct 06, 2017 11:15 am

Avs saving utf8

Post by Dimsok » Fri Oct 06, 2017 11:23 am

Comskip make avs in utf8 (65001), which unsupport by avisynth. I have to manually change encoding to ansi 1251 to run it. Is there a way to make it auto?

erik
Site Admin
Posts: 3310
Joined: Sun Aug 21, 2005 3:49 pm

Re: Avs saving utf8

Post by erik » Fri Oct 06, 2017 11:25 am

Is that the filename?

Dimsok
Posts: 9
Joined: Fri Oct 06, 2017 11:15 am

Re: Avs saving utf8

Post by Dimsok » Fri Oct 06, 2017 11:56 am

It's Cyrillic name. For example, Первый канал.TS.

erik
Site Admin
Posts: 3310
Joined: Sun Aug 21, 2005 3:49 pm

Re: Avs saving utf8

Post by erik » Fri Oct 06, 2017 12:14 pm

I was already happy to get it working for utf8, not sure I want to change it.....

Dimsok
Posts: 9
Joined: Fri Oct 06, 2017 11:15 am

Re: Avs saving utf8

Post by Dimsok » Fri Oct 06, 2017 12:26 pm

I did try on file 22.ts, and avs was made in 1251. so it started normally. Why to use utf8 on cyrillic, if it can be in ansi (without any umlauts. accent-graves, kanji, chinese characters etc).

erik
Site Admin
Posts: 3310
Joined: Sun Aug 21, 2005 3:49 pm

Re: Avs saving utf8

Post by erik » Fri Oct 06, 2017 12:27 pm

Can you send me a very short recording with filename in ANSI 1251?

Dimsok
Posts: 9
Joined: Fri Oct 06, 2017 11:15 am

Re: Avs saving utf8

Post by Dimsok » Fri Oct 06, 2017 12:57 pm

Sent to pm.

Jagad
Posts: 130
Joined: Wed Dec 07, 2011 12:37 am

Re: Avs saving utf8

Post by Jagad » Fri Oct 06, 2017 2:25 pm

erik wrote:I was already happy to get it working for utf8, not sure I want to change it.....
It does not make any sense changing back to old outdated encoding. I hope you not even contemplating it. This is not a comskip problem, but user have to solve it by himself if he wants to use ancient software. VRD will finally support Unicode (utf-8) fully in next version (v6 release in a few months).
Dimsok wrote:Comskip make avs in utf8 (65001), which unsupport by avisynth. I have to manually change encoding to ansi 1251 to run it. Is there a way to make it auto?
You will have to use a conversion script like the rest of us if you want to go from Unicode (utf-8) to ANSI automatically. Google it and you will find lots of em.

erik
Site Admin
Posts: 3310
Joined: Sun Aug 21, 2005 3:49 pm

Re: Avs saving utf8

Post by erik » Fri Oct 06, 2017 3:00 pm

Indeed I will not change as I simply don't know how to make this work for all test files I have.

Jagad
Posts: 130
Joined: Wed Dec 07, 2011 12:37 am

Re: Avs saving utf8

Post by Jagad » Fri Oct 06, 2017 3:18 pm

Puh, thanks erik :D I have waited 6 long years on this VRD unicodification and im very eager to strip away all my conversionscripts from my automated commercialstipping file-editing system...going pure Unicode. Was shoaking on my coffee there for a while, but im relaxed again :) :lol:

popy
Posts: 4
Joined: Wed Sep 03, 2014 7:17 pm

Re: Avs saving utf8

Post by popy » Sun Apr 01, 2018 2:42 pm

Hey.

Also have an issue with recordings from DVBV which have UTF8 chars in it.
Comskip says: could not open file.

Here is a link to an issue i have created: https://github.com/erikkaashoek/Comskip/issues/101

Is there anything i am missing :D

thx
pOpY

erik
Site Admin
Posts: 3310
Joined: Sun Aug 21, 2005 3:49 pm

Re: Avs saving utf8

Post by erik » Sun Apr 01, 2018 2:53 pm

What file extension?

popy
Posts: 4
Joined: Wed Sep 03, 2014 7:17 pm

Re: Avs saving utf8

Post by popy » Sun Apr 01, 2018 2:58 pm

Here is the filename which fails:

2018-03-31_22-00-00_ProSieben MAXX_60 Days In – Undercover im Knast - Schlangennest.ts

And one which works:

2018-03-30_00-05-00_RTL NITRO_MythBusters The Search - Der Fast & Furious -Schleudersitz.ts

Can you see the larger – sign in the first file name, when i'll remove this, the issue is gone.
It's an UTF8 char (EN DASH) 0xE2 0x80 0x93 (e28093)

I have made progress here, when ill open a cmd window, set the font to Lucida console and executing "chcp 65001" i am able to specify the filename to comskip and it works.
Now an recording is running in DVBV with an unicode sign in filename and after that my post_recording.bat (which calls comskip) will be executed.

Hope this works.

Or do you have an guide for DVBV + UTF8 chars + comskip?

thx
pOpY

popy
Posts: 4
Joined: Wed Sep 03, 2014 7:17 pm

Re: Avs saving utf8

Post by popy » Sun Apr 01, 2018 6:58 pm

Update:

It work now with an batch file!
In the batch i had:

comskip.exe "%1" -t

but it should be:

comskip.exe %1 -t

Also when you copy the filename of an file from explorer which has UTF8 chars in it into an CMD window the UTF8 chars will be converted to ASCII from windows and comskip does'nt find it.
So my issue is solved.

pOpY

erik
Site Admin
Posts: 3310
Joined: Sun Aug 21, 2005 3:49 pm

Re: Avs saving utf8

Post by erik » Sun Apr 01, 2018 7:16 pm

Happy to hear

Post Reply