Basically, you can't, not with a byte array.
You can do it for objects larger than 2GB, if you are:
1) Running in 64bit mode on a 64 bit system. 32bit apps cannot address memory bigger than 2GB.
2) Are running .NET Framework V4.5 or greater.
3) Have set gcAllowVeryLargeObjects in your app.config: gcAllowVeryLargeObjects Element
But ... array indexers are still limited to an integer - so you can't do this with byte arrays as your array index would be too big.
You can do it with a stream (~8TB is allowed) but not with an array.
And frankly, if you need to load a single file that large into memory, you probably need to look at how your app is working anyway because there is probably something very wrong with your basic algorithm...