(See also this question, which is similar, but talks about the situation where the language is normally stored on disk with one command per byte.)
This question originally came up in connection with 7, but might be relevant to other languages.
In 7, each command is only three bits long (it only has 8 commands which can be specified in source files). It accepts multiple input formats; one uses one byte per command (for ease of editing and testing), one reads the sequence of bits formed by the input file and interprets every three bits as a command. Because most computer systems don't allow a file to contain a fractional number of bytes, any amount of "trailing whitespace" (in this case, 1 bits) can be placed at the end of a file, and is ignored by the interpreter's file reading routines.
The information content of the language is obviously 3 bits (i.e. ⅜ of a byte) per command. Dennis has suggested that the length of a program should be rounded up to a whole number of bytes, because that's how file storage works. Alternatively, you could the length down by removing trailing 1 bits (as the program considers them not to exist). How should this be counted?
As a final question to consider: if the interpreter was changed to accept programs over an RS-232 interface (which is physically capable of sending an arbitrary number of bits), would that change the scoring?