Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cx_Oracle 8.0.1 - `DPI-1057: buffer size too large` inserting into a CLOB #480

Closed
Fxztam opened this issue Sep 27, 2020 · 4 comments
Closed

Cx_Oracle 8.0.1 - `DPI-1057: buffer size too large` inserting into a CLOB #480

Fxztam opened this issue Sep 27, 2020 · 4 comments
Labels

Comments

@Fxztam
Copy link

@Fxztam Fxztam commented Sep 27, 2020

Hello,

I am trying to load GeoJSON data with Python37 & cx_Oracle 8.0.1 into my ALways Free 19c OracleATP (Frankfurt)
with module: load.load2table.py from my Ring of Oracle GeoJSON : https://github.com/Fxztam/roogeos and
get:

_$$$ Error load.load2table(insert into ldgeojson) : DPI-1057: buffer size of 1964808393 is too large (max 1073741822) Traceback (most recent call last): File "..\orageojson\load\load2table.py", line 23, in load2table values ( :clobdata )''', clobdata=geojson) cx_Oracle.DatabaseError: DPI-1057: buffer size of 1964808393 is too large (max 1073741822)_

Is there a way or good tip to solve it?

Thanks and regards,
Friedhold

@Fxztam Fxztam added the bug label Sep 27, 2020
@cjbj
Copy link
Member

@cjbj cjbj commented Sep 27, 2020

Can you help us by giving more context - and a runnable example (with less data!)? Where/what is load2table? https://github.com/Fxztam/roogeos/search?q=load2table.py doesn't help, and neither does a google search.

@Fxztam
Copy link
Author

@Fxztam Fxztam commented Oct 4, 2020

Sorry for the delay:

The GeoJSON file length is 1,964,808,393 bytes (and I cannot cut it) and it means the:

cx_Oracle.DatabaseError: DPI-1057: buffer size of 1,964,808,393 is too large (max 1,073,741,822)

The code source is from here: https://github.com/Fxztam/roogeos/blob/master/RingOfOraGeo/orageojson/load/load2table.py

    p_cursor.execute('''insert into ld_geojson (ldjson)  values ( :clobdata )''', clobdata=geojson)

so I cannot load this large GeoJSON Feature Collection into OracleATP CLOB with Cx_Oracle .. ?

Thanks and regards,
Friedhold

@cjbj cjbj added the question label Oct 5, 2020
@cjbj
Copy link
Member

@cjbj cjbj commented Oct 5, 2020

To insert (or read) data over 1 GB with LOB columns, you need to stream the data in chunks, see Streaming LOBs (Write).

@cjbj cjbj changed the title Cx_Oracle 8.0.1 - `DPI-1057: buffer size of 1964808393 is too large` Cx_Oracle 8.0.1 - `DPI-1057: buffer size too large` inserting into a CLOB Oct 5, 2020
@cjbj cjbj removed the bug label Oct 5, 2020
@Fxztam
Copy link
Author

@Fxztam Fxztam commented Oct 5, 2020

Thanks, I will try it.

@Fxztam Fxztam closed this Oct 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.