arcpy.da.NumPyArrayToTable() RuntimeError: Unspecified error

by Son of a Beach   Last Updated July 12, 2019 01:22 AM - source

Using arcpy for ArcGIS Desktop 10.7.1...

In order to append data from a Pandas data frame to an ArcGIS table, I am attempting to create a temporary in-memory table from the Pandas data frame as follows:

tmpTable = "in_memory/tmpTable"
labResults = pandas.read_csv(csvPath, parse_dates=[3], dayfirst=True)

######  If this ONE next line is commented out and the following line is UNCOMMENTED, the error does NOT occur
labResults['ResultQualifier'] = np.where(labResults.Result.str[0].str.isdigit(), None, labResults.Result.str[0])
#labResults['ResultQualifier'] = np.where(labResults.Result.str[0].str.isdigit(), labResults.Result.str[0], labResults.Result.str[0])

labResults['ResultValue'] = pandas.to_numeric(np.where(labResults.Result.str[0].str.isdigit(), labResults.Result, np.where(labResults.Result.str[1].str.isdigit(), 
labResults.Result.str[1:], labResults.Result.str[2:])))
del labResults['Result']
labResults.rename(columns={'ResultValue': 'Result'}, inplace=True)

arr = np.array(np.rec.fromrecords(labResults.values))
arr.dtype.names = tuple(labResults.dtypes.index.tolist())
arcpy.da.NumPyArrayToTable(arr, tmpTable)
######  ERROR OCCURS on this next line
arcpy.Append_management(tmpTable, resultsTable, "NO_TEST")

But the arcpy.da.NumPyArrayToTable() fails with the frustratingly generic error:

`RuntimeError: Unspecified error`

I can print the data frame by entering labResults on the Python console, and it prints out the expected data. And labResults.dtypes produces the expected output also, so I'm confident that the data frame is all OK.

NB: I get the same error whether tmpTable is a fGDB table (fGDB exists, table does not) or an in_memory table.

How can I get this to work as expected?

Note: This code is based on: where there is a full explanation of what it all does.


90529,366330,"site1",15/1/19 10:15:00,"Glyphosate and Glufosinate in Water by LC-MS/MS","Glyphosate","µg/L","<5"
90529,366330,"site1",15/1/19 10:15:00,"Solids, Total Suspended dried at 104 ± 3 °C","TSS","mg/L","21"
90529,366331,"site2",15/1/19 10:35:00,"Glyphosate and Glufosinate in Water by LC-MS/MS","Glyphosate","µg/L","<5"
90529,366331,"site2",15/1/19 10:35:00,"Solids, Total Suspended dried at 104 ± 3 °C","TSS","mg/L","577"
90529,366332,"site3",15/1/19 11:00:00,"Glyphosate and Glufosinate in Water by LC-MS/MS","Glyphosate","µg/L","<5"


I have found that if I change the labResults['ResultQualifier'] = line to never include NULLs (None's), then the error does NOT occur. This line is there to create a new column in the data frame that includes either < if the first character of 'Result' is not a digit, or otherwise 'None' (NULL). I'm not sure why NULL values would be causing such a problem.

Related Questions

Iterate over specific columns by name in Python

Updated July 23, 2018 15:22 PM