API - REST or SOAP - Bulk Create Contacts

Hello,

My requirement is to fetch data from CSV file and create contacts.

Version:

OSvC: February 2016

CPHP: V1_3

Rows in CSV file: 99500

I have build a custom script which process all rows from CSV file and create contacts using Connect PHP, but here I am getting memory error:

"DB API Error SQL Fcn: execute() [SQLStatement.cpp: 641] 
       Error: Unable to allocate sufficient storage for queryPointer!"

After 4000 rows, script stops automatically.

Can i use REST/SOAP API to create bulk contacts? Please provide example.

Thanks,
Monish

Comments

Hi Monish,

If this does not need to be a recurring automated process, you can use the standard Data Import Wizard in the console to upload bulk data from CSV and import into almost any object.

Otherwise, as you've found, ConnectPHP is plagued with memory issues (even in the newer versions). Without setting up a clunky auto-restart process, I'd recommend writing a small external app that uses the SOAP API to create contacts. Here is a link to a documentation for your version. You'll want to look at the Contact primary object, the Create API operation, and the Batch API operation; there are some basic examples in the documentation.

To my knowledge, the REST API does not support batch create operations at this time, so you'd need to make a separate HTTP request for every contact, which would be terribly inefficient, so this is probably not a good option.

Hi Ben,

Thanks for your reply!

Data Import Wizard would not make sense in my case as I am trying to insert contacts with some customization.

I have started with SOAP API with Create Operation and Batch API Operation.

Any idea about, where I will get WSDL file only for Create Contact Operation and Batch Operation, so that I can prepare proper xml soap request with my customization ?

Thanks for your help!

Regards,
Monish

Hi Monish,

The standard typed WSDL is located at https://<your_site>/cgi-bin/<your_interface>.cfg/services/soap?wsdl=typed. It includes the entirety of the typed SOAP API definition; Oracle doesn't publish separate, smaller WSDLs for specific operations. The Getting Started section of the documentation contains step-by-step instructions for getting setup in .NET and Java environments.

Thanks, and good luck!

Ben is right that Connect PHP can have some bad memory problems. I remember triaging them once and they seemed to be related to the number of custom fields on objects, but that was a few years ago.

Your particular error does look to be something different. Are you calling commit explicitly at all in your script. If you aren't you are probably overloading the commit buffer in the DB.

Commit can be explicitly called via this code (assuming you aliased the Connect PHP API to RNCPHP):

RNCPHP\ConnectAPI::commit();

You can play with the interval, but I usually do a commit every 100 records or so.

Hi Rowse,

Thanks for reply!

Yes, I am calling commit statement after every 1000 records processed.

Commit Statement:
RNCPHP\ConnectAPI::commit(RNCPHP\RNObject::SuppressAll);

I am also processing some custom fields while import for per contact.

As Ben suggested, I am almost done with SOAP API method, facing some issues but will update once done.

Regards,
Monish

Zircon - This is a contributing Drupal Theme
Design by WeebPal.