Using pandas' to_sql()
Attempting to upload large dataframes can result in the following error:
PendingRollbackError: Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b)
This can occur even where max log IO percentage is less than 100%. E.g.:
Setting a chunksize
- e.g. chunksize=50000
- can get around this.
This post is licensed under CC BY 4.0 by the author.