Garmaine Staff asked 2 years ago

My goal: I need to grab N number of files from an FTP, parse them, and send the parsed data somewhere. My current implementation is working fine but when I get past the first file it begins to hang.

The package I am using for the csv stream / parse is fast-csv. Below is an example of what I was trying to accomplish. It only ever runs 1 time. I have a feeling it has something to do with the stream not ending or finishing properly. I was hoping someone could give me some pointers on a direction I could take. Let me know if you need anymore clarification.

async handleStreams (files) {
  for await (const file of files) {
    await this.func(file)
  }
}

async func (file) {
    return new Promise((resolve, reject) => {
    const stream = readableStream(file)
    stream
      .on('error', error => { 

      })
      .on('data', (row: string[]) => {
        // do stuff
                stream.end()
      })
      .on('end', async (rowCount: Number) => { 
        await sendOff()
        // resolve()????
      })
      .on('close', async () => {
                // resolve()????
      })
  })  
}