# Working With Multiple Files

**URL:** <https://forum.cloverdx.com/t/working-with-multiple-files/1930>\
**Category:** CloverDX Platform\
**Created:** [January 17, 2018, 12:00am UTC](https://forum.cloverdx.com/t/working-with-multiple-files/1930 "2018-01-17T00:00:00Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![jcatoe](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@jcatoe](https://forum.cloverdx.com/u/jcatoe)\
**Post date:** [January 17, 2018, 12:00am UTC](https://forum.cloverdx.com/t/working-with-multiple-files/1930/1 "2018-01-17T00:00:00Z")

</div>

I’m trying to replicate this process with CloverETL:

- download multiple zip files from a website  
- unzip the text files within; each has the same name as the zip file they’re in  
- the text files get converted into tables within a Postgres database  
- then some tables get joined together

My question is: do I need to create multiple paths for each file or can they be transformed together without explicitly referencing individual files?

I have successfully been able to download all of the zipped files from the website by reading in a CSV of filenames and mapping it through the HTTPConnector, but from there, I don’t know how to handle the different metadata for each file without working with each file separately.

I’m hesitant to work with them separately because there are 20-30 files and new ones get generated each year so it would require yearly maintenance that I may not be here for and it would get unwieldy quickly.

I was hoping that [http://forum.cloveretl.com/viewtopic.php?t=3622](http://forum.cloveretl.com/viewtopic.php?t=3622) would help me, but it was written in CLT1, and I haven’t been able to replicate it successfully.

Thank you for the help!

---

<div class="post-metadata">

**Author:** ![the\_goat](https://avatars.discourse-cdn.com/v4/letter/t/f475e1/32.png) [@the\_goat](https://forum.cloverdx.com/u/the_goat)\
**Post date:** [January 18, 2018, 2:41pm UTC](https://forum.cloverdx.com/t/working-with-multiple-files/1930/2 "2018-01-18T14:41:29Z")

</div>

Are we talking about some BLOBs or CLOBs in that database? Or as I suspect, you need to have them parsed into fields/columns and then put to the database in some sort of format?

---

<div class="post-metadata">

**Author:** ![jcatoe](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@jcatoe](https://forum.cloverdx.com/u/jcatoe)\
**Post date:** [January 19, 2018, 3:51pm UTC](https://forum.cloverdx.com/t/working-with-multiple-files/1930/3 "2018-01-19T15:51:53Z")

</div>

Columns and tables. And we really just drop all of the tables and recreate them from the text files for all but one table.

---

<div class="post-metadata">

**Author:** ![PedroVazquezRosario](https://avatars.discourse-cdn.com/v4/letter/p/ed8c4c/32.png) [@PedroVazquezRosario](https://forum.cloverdx.com/u/PedroVazquezRosario)\
**Post date:** [January 23, 2018, 5:16pm UTC](https://forum.cloverdx.com/t/working-with-multiple-files/1930/4 "2018-01-23T17:16:19Z")

</div>

I created an example of how you can achieve your use case. I used a combination of Jobflows and Graphs to generate the metadata, and the database tables dynamically based on the data in each file and then inserted the data based on the table name.

- [MultipleFilesExample.zip](https://cdn.cloverdx.com/forum/18200%5F2d0bc68304c1f07ad9487b0a790317b9/MultipleFilesExample.zip)