Coming from a C# golfing background, as far as I'm concerned, as long as csc compiles it without complaining, then it's not a problem, and you don't need any
import clauses in your code at all - they are not the references, they are just for naming's sake. The references are indeed usually sorted out by the csproj/vbproj, but mscorlib (which contains just about anything you'd ever want for golf) is included by default (as I recall, if you look in Visual Studio, there is a specific project option somewhere to not include mscorlib, because it's a special command for csc/vbc (mono versions exist, as Peter Taylor has said)).
For example, this simple VB.NET program compiles fine by just passing it to vbc (e.g.
Note the lack of any
System in there at all.
I have supplied C# code for challenges before, and supplied instructions to compile them with csc (I don't provide a csproj or the like), which I have reason to believe has been done.
As long as you don't use anything outside of mscorlib, then your answer is going to be valid. It is a feature of VB.NET that it imports
System by default (whether the vbproj does or not), just as it is a feature of C# that is doesn't, and exploiting either is fine. If you are ever worried about the validity of your code, then just throw it at vbc, and if it compiles, then as far as I'm concerned, no one has any reason to complain about it.
In The Example...
Whether or not the example's byte score is valid is not something I would wish to comment on, but I would suggest providing the command to compile any program that isn't absolutely trivial so that people can make up their own mind.
In the example you provided, however, you need a references to
System.Numeric.dll, which you would have to tell to the compile (with a
/r:System.Numeric.dll command-line argument). I don't know how this would be counted (I prefer not to, and have never needed to, use any external libraries before - it's usually more fun to not do so).
Part of the pain is that
vbc test.vb /r:System.Numerics.dll isn't going to work from most directories - the compiler needs to also know where to find said DLL. Like I said, I steer clear of using non-mscorlib libraries for a host of reasons, this one inclusive. I imagine though that you'd be allowed to assume it's in the same directory, given we assume the code is in the working directory anyway, this would seem fair.
Just occurred to me that most people aren't going to bother to run csc or vbc; if anything, they will put the code into IDEOne, or something similar. In the case of the example it fails to compile, complaining about being unable to access System.Numerics.BitInteger.
My stupid example program, however, runs fine on IDE-One, as will just about any other piece of .NET golf you find on this site.
The only time I've ever shipped a csproj on this site was when I was providing a library for other people to use in the Genetic Algorithm competition for which byte count didn't matter (and that was only because I had more than one file).