Question: Issues importing large datasets

I’ve been having some issues working with large datasets / matrixes in maple 17.02 and 2015. My data consists of a 10^7 x 14 csv file with several lines of header information. Attached is a small sample. The ImportData assistant hangs while importing said file. The javaw process stops responding for a period of time then stops consuming cpu time. I’ve have successfully imported a file of the same format but reduced in size (10^6 x 14) with this same function. So I don’t believe it’s a formatting issue but rather its size.

Are there size limitations to the ImportData function?

The attached maple file has a test case in which the data set (sans header info) is created and exported as a csv file. The export time took longer than I expected (~2 hrs). I then attempted to import the file using two different functions. The ImportMatrix function successfully imported the test case file in approximately 20 minutes, however the ImportData functions seems to fail in the same way as it does importing my actual dataset. I haven’t successfully used the ImportMatrix function on my actual dataset; I’m assuming the header information is the source of the problem.

Are there other methods to import this data?

As stated above, I’m tried both maple 17 and 2015 both 64 bit versions running on an Intel i7 M620 @ 2.67Ghz, 8 GB ram (~ 6 GB avail), sata 2 ssd.

Thank you,

Ron

importtest.mw  Sample.txt