How to write a large CSV file in Node.js
How to write a large CSV file in Node.js
Today, we will show you how to write a large CSV file in Node.js. Suppose you want to generate a CSV file using a dummy record in Node.js. Here we will show you a simple example to generate a large CSV file.
Checkout more articles on Node.js
Solution
Use the following code to generate the large CSV file using Node.js. Here we have used the fs.createWriteStream to create a CSV file.
1 2 3 4 5 6 7 8 9 10 11 | var fs = require('fs'); // ... // ... const fileName = "product_10L.csv"; const fileWriteStream = fs.createWriteStream(fileName); for (let i = 1; i <= 1000000; i++) { const ableToWrite = fileWriteStream.write(`Product_${i},SKU${i}\n`); if (!ableToWrite) { await new Promise(resolve => fileWriteStream.once('drain', resolve)); } } |
In the above code, we are trying to write a dummy product CSV file that contains the product name and product SKU. The CSV file will be generated at the root level of the project.
API Example
Your server.js file should look like below if you add this logic to the rest API.
Refer this article to setup REST API in node.js.
Create REST API in Node.js
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | var express = require('express'), app = express(), port = process.env.PORT || 4000; var fs = require('fs'); app.get('/write-csv', async (req, res) => { const { size } = req.query; const fileName = "product_10L.csv"; const fileWriteStream = fs.createWriteStream(fileName); for (let i = 1; i <= size; i++) { const ableToWrite = fileWriteStream.write(`Product_${i},SKU${i}\n`); if (!ableToWrite) { await new Promise(resolve => fileWriteStream.once('drain', resolve)); } } res.send({ success: true, message: 'File generated successfully.' }); }); app.listen(port, () => { console.log('Server started on: ' + port); }); |
Here, we have passed the number of data as a query parameter (?size=1000000). Run the following command to write 10 Lakhs records in the CSV file and the generated file will be available at the root level.
1 | http://localhost:4000/write-csv?size=1000000 |
I hope you find this article helpful.
Thank you for reading. Happy Coding..!! 🙂
