iconv: file too large

The iconv utility is used to convert file encodings. I'm using it to convert a Postgresql database from LATIN1 to UTF8.

However, the standard iconv program slurps the entire file into memory, which doesn't work for large data sets (such as database exports). You'll see errors like:

iconv: unable to allocate buffer for input: Cannot allocate memory
iconv: cannot open input file `database.txt': File too large

This script is just a wrapper that processes the input file in manageable chunks and writes it to standard output: iconv-chunks