index > Common Language Runtime > Failed to initialize huge arrays

Failed to initialize huge arrays

I have to initialize a huge constant array of structures(60000~100000),
used as a lookup table , but it failed with System.InvalidProgramException
when the items reach 18400. and if I change the item type from struct to class,
the threshold is 35000 . It seems that there a size limition when initialize constant in CLR.
Is there any configurable parameters can be used to increase this limition in C# compiler
or .NET framework. Or I'm missing something there?

using System;

struct st
{
public short t1,t2 ;
public st(short t1,short t2){
this.sym = t1;
this.act = t2;
}
}

class Test{
static void Main(){

     st[] a = {
      new st(3,4),
      new st(3,4),
      new st(3,4),
      new st(3,4),
      new st(3,4),
      new st(3,4),
      new st(1,2),
         ...
         ...
   repeated more than 20000 times
         ...
         ...
         ...
      new st(3,4),
      new st(1,2),
      new st(1,2),
      new st(3,4),
      new st(1,2),
      new st(1,2),
      new st(3,4)
     };
     Console.WriteLine("{0}",a[1].t1);
}
}

damaha

>It seems that there a size limition when initialize constant in CLR.

This isn't a constant from the CLR's point of view. The C# compiler rewrites the code to something like

st[] a = new st[20000];
a[0] = new st(3,4);
... and so on ...

I think it's more likely that the method body ends up being too big or something. This really isn't the most efficient way to accomplish what you want. If there's some pattern to when you use 1,2 or 3,4, you could rewrite the code as

st[] a = new st[20000];
for (int i = 0; i < st.Length; i++) {
if (some condition based on i)
a[ i ] = new st(1,2);
else
a[ i ] = new st(3,4);
}

If the data is more random, I suggest you store it as a binary blob in an embedded resource file and load it dynamically from there.

Mattias Sjögren

Thanks for your help.
It seems a good solution to use embedded resource file to manage this big arrays.

But there is another problem when getting this array back from resource file.

there is always a System.InvalidCastException for this line:

st[] a = (st[]) rm.GetObject("array 1",ci);

can anyone help?

It's ok to create a resource file using following code:

using System;
using System.Resources;

[Serializable]
struct st
{
public short t1,t2 ;
public st(short t1,short t2){
this.t1 = t1;
this.t2 = t2;
}
}

public class WriteResources {

public static void Main(string[] args) {

st[] a = {
   new st(3,4),
   new st(3,4),
   new st(3,4),
   new st(3,4),
   new st(3,4),
   new st(3,4)
};

      // Creates a resource writer.
      IResourceWriter writer = new ResourceWriter("myResources.resources");

      // Adds resources to the resource writer.
      writer.AddResource("String 1", "First String");

      writer.AddResource("array 1", a);

      writer.Generate();

      // Writes the resources to the file or stream, and closes it.
      writer.Close();
   }
}

but failed when retrieve array back from the resource file,

using System;
using System.Resources;
using System.Collections;
using System.Globalization;
using System.Threading;
using System.Reflection;

[Serializable]
struct st
{
public short t1,t2 ;
public st(short t1,short t2){
this.t1 = t1;
this.t2 = t2;
}
}

class EnumerateResources
{

public static void Main()
{

  // Create a resource manager to retrieve resources.
        ResourceManager rm = new ResourceManager("myResources",
          Assembly.GetExecutingAssembly());

CultureInfo ci = Thread.CurrentThread.CurrentCulture;

//It's ok to retrieve string
String str = rm.GetString("String 1", ci);
Console.WriteLine(str);

//THERE IS ALWAYS A System.InvalidCastException for this line:
st[] a = (st[]) rm.GetObject("array 1",ci);
Console.WriteLine(a[1].t1);
}
}

damaha

You've defined your structure in two different assemblies, once in the resource writer assembly and once in the resource reader assembly. To the CLR this means that they're different, even though they have the same name and same layout. To fix this you'll need to put your S structure in a .dll that both programs reference.

-Shawn

Shawn Farkas - MS

Mattias Sjögren

Thanks for your help.
It seems a good solution to use embedded resource file to manage this big arrays.

But there is another problem when getting this array back from resource file.

there is always a System.InvalidCastException for this line:

st[] a = (st[]) rm.GetObject("array 1",ci);

can anyone help?

It's ok to create a resource file using following code:

using System;
using System.Resources;

[Serializable]
struct st
{
public short t1,t2 ;
public st(short t1,short t2){
this.t1 = t1;
this.t2 = t2;
}
}

public class WriteResources {

public static void Main(string[] args) {

st[] a = {
   new st(3,4),
   new st(3,4),
   new st(3,4),
   new st(3,4),
   new st(3,4),
   new st(3,4)
};

      writer.AddResource("array 1", a);

      writer.Generate();

      // Writes the resources to the file or stream, and closes it.
      writer.Close();
   }
}

but failed when retrieve array back from the resource file,

using System;
using System.Resources;
using System.Collections;
using System.Globalization;
using System.Threading;
using System.Reflection;

[Serializable]
struct st
{
public short t1,t2 ;
public st(short t1,short t2){
this.t1 = t1;
this.t2 = t2;
}
}

class EnumerateResources
{

public static void Main()
{

  // Create a resource manager to retrieve resources.
        ResourceManager rm = new ResourceManager("myResources",
          Assembly.GetExecutingAssembly());

CultureInfo ci = Thread.CurrentThread.CurrentCulture;

//It's ok to retrieve string
String str = rm.GetString("String 1", ci);
Console.WriteLine(str);

//THERE IS ALWAYS A System.InvalidCastException for this line:
st[] a = (st[]) rm.GetObject("array 1",ci);
Console.WriteLine(a[1].t1);
}
}

damaha

-Shawn

Shawn Farkas - MS