Tweetable Video
code-challenge printable-ascii compression image-processing
This challenge is inspired by the Encode Images into Tweets challenge, extending the idea to now attempt to compress video into a series of "tweet" sized messages where the number of tweets == number frames of the video.
The competition is to compress 450 frames of video into 450 (or less) tweets of 140 charactes ( 114.97 bytes per tweet, total max 51737 bytes).
Original frames as Video, as PNG images [Sandbox note: Higher quality version will be uploaded soon]
Reference solution frames as Video [Sandbox note: Reference solution will be expanded once challenge changes stabilized]
Rules:
These rules are the based from rules in Encode Images into Tweets modified to accommodate multiple frames that will yield the resultant video.
- You must write a program that will communicate via std in/out based on the communication protocol defined below, ultimately producing a series of tweets that will, when fed back to your program, produce image frames of the video.
- The text created for each 'Tweet' by the program must be at most 140 characters long and must only contain characters whose code points are in the range of 32-126, inclusive.
- The source video will be 450 frames of Big Buck Bunny. Frames can be downloaded here
- Frames will be width 320 pixels and height 240 pixels @ 24 bit RGB, each channel having 8 bit precision.
- Your program should be able to compress other video sources at same resolution.
- Your program can use external libraries and files, but cannot require an internet connection or a connection to other computers.
- The decoding process cannot access or contain the original images in any way. (Clarification use of corpus images is allowed as long as these images do not contain direct or manipulated original image frames)
- Use of video codecs or existing libraries that perform video compression are not allowed. This is to encourage experimentation and originality rather than simply calling libraries. Note that use of static image compressors/ libraries are allowed.
- Standard Loopholes are forbidden.
Communication Protocol
- All communication is via std in/out.
- The controller program will launch your program with no command line arguments.
- The controller will either initiate a one of the following sequences commanding to your program and expecting results.
Note the sequences are described below in the following format:
<Comms direction from controller perspective>, <seq number>: <Transmitted Data> - <Information>
For Encoding of the frames:
Output, 1: ENCODE\n - Indicating that the controller will be sending frames for your program to compress.
Output, 2: <number of frames>\n - String indicating number of frames. i.e. 450
Output, 3: <number of bytes for image data>\n - String indicating number of bytes of the current image data. e.g. 32145
Output, 4: <image data> - sequence of bytes of length defined in output line 3. The is the image data. The image data is a 24 bit image in PNG format
Repeat from output seq number 3 for the number of frames in output seq number 2.
Input, 1: <Tweet>\n - encoded image data. upto 140 characters, whose code points are in the range of 32-126, inclusive. A zero sized tweet will indicate end of all tweets.
Repeat from input line 1 for 1 to up to the number of frames in output seq number 2. Note a zero sized tweet will indicate end of all tweets.
Output, 5: END\n - Indicating that all processing has finished and your program should quit gracefully.
For Decoding of the frames:
Output, 1: DECODE\n - Indicating that the controller will be sending tweets for your program to decompress.
Output, 2: <number of tweets>\n - String indicating number of tweets. i.e. 450
Output, 3: <Tweet>\n - encoded image data. upto 140 characters, whose code points are in the range of 32-126
Input, 1: <number image frames decoded>\n - String indicating number of image frames that have been decoded as a result of the provided tweet. If 0 then no images have been decoded for this tweet.
Input, 2: <number of bytes for image data>\n - String indicating number of bytes of the current image data. e.g. 32145.
Input, 3: <image data> - sequence of bytes of length defined in input line 2. The is the image data. The image data is a 24 bit image in PNG format.
Repeat from input seq number 2 for the number of frames in input seq number 1.
Repeat from output seq number 3 for the number of tweets in output seq number 2.
Output, 4: END\n - Indicating that all processing has finished and your program should quit gracefully.
- Your program will be restart between the encoding and decoding phases.
Scoring
As an attempt to model the human visual system in a simplistic manner, JPEG quality levels are used as an analogue. Each frame will be compressed to a JPEG using quality levels 0.01 to 1.0 in 0.01 increments. The PSNR of these JPEG compressed versions are calculated and added into a data structure. Additionally, each frame will be resized by half and the JPEG encoding and PSNR calculations performed and added to the data structure. The resizing and calculating will repeat until a single pixel is left.
The result of which will be a scoring continuum of visual 'quality' from low (single color frame) to High (near-identical to original).
Thus, scoring will be objective by means of the following formula:
JPEG_QUALITY_LEVEL_PSNR = a pre-computed data multi dimensional structure consisting of the
PSNR for incremental JPEG quality levels for each image frame of
the video. The entries into this structure for each frame is such
that higher indecies are deemed to be of more similar to the
original.
Score = SUM of (index of closest matching entry in JPEG_QUALITY_LEVEL_PSNR for given
original frame and given PSNR of decoded frame)
Controller
[Sandbox note: Controller is to be created]
Refrence Solution
[Sandbox note: Reference is to be created once question stabilizes]
Sandbox Questions
Ok, so this is my first question ever posed to codegolf and I am very keen to get feedback on how to better present the challenge.
Some areas where I would like specific feed back is the objective scoring mechanism... i purposely avoided direct PSNR as the score as it is very limited for scoring video quality, being biased toward blurry images, however other algorithms are too complex or not defined. So as a compromise I am using JPEG compression as an analogue for the Human Visual System. Effectively matching a given solutions output frame to original frame PSNR to a JPEG compressed, at a given quality level) version's PSNR and using that match's index in sorted JPEG quality order as input for the score.
Is the protocol clear enough? if not what would be a better way to present it?