how to handle large flat files?

1. pub.xmlmlStringToXMLNode (Input: $fileStream)
2. pub.xml:getXMLNodeIterator (Input: "node" from previous step, "criteria" - records which are multiple)
3. Repeat step ( Until data exist)
3(a) pub.xml:getNextXMLNode (Input: "iterator" from Step 2)
3(b) Map data to flat file
3(c) pub.flatFile:convertToString
3(d) writeToFile

this is one of the way to handle large documents

TN for Large Document handling tn.BigDocThreshold
---------------------
-n: Specify a negative number to have Trading Networks
consider no documents as large.

0:all documents as large
n: no of bytes greater the specified n value

in integrationserver also you have watt.server.tspace property where you can define maximum num of bytes
that can be stored on hard disk.. not much clear about this property.. but check

..tspace is nothing but TN specific naming convention for Disk, this property suggest size of disk allocated to store all Large doc coming to TN for processing..!!

First save the flat file in ur system disk memory and call convertTovalues service and in that service enable the iterator attribute.....For more info just read flat file developer guide for handling large flat files