Skip to content

/arrow

Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational libraries and zero-copy streaming messaging and interprocess communication…
arrow
C++ Java Rust Python TypeScript Ruby Other
  1. C++ 44.0%
  2. Java 14.4%
  3. Rust 9.2%
  4. Python 9.1%
  5. TypeScript 4.5%
  6. Ruby 3.5%
  7. Other 15.3%
Branch: master
Find file
Clone or download

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Open in Desktop Download ZIP

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio

If nothing happens, download the GitHub extension for Visual Studio and try again.

@liyafan82 @lidavidm
liyafan82 and lidavidm ARROW-7437: [Java] ReadChannel#readFully does not set writer index co… 
…rrectly

1. The writer index should be incremented by the amount of data actually read.
2. When EOS is encounterned, the number of bytes read should be incremented before returning.

Closes #6064 from liyafan82/fly_1219_idx and squashes the following commits:

e268b0d <liyafan82>  Resolve comments
baab805 <liyafan82>  ReadChannel#readFully does not set writer index correctly

Authored-by: liyafan82 <fan_li_ya@foxmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
Latest commit 81f0d42 Dec 27, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github ARROW-6742: [C++] Remove boost::filesystem dependency in hdfs_interna… Dec 19, 2019
c_glib ARROW-7420: [C++] Migrate tensor related APIs to Result-returning ver… Dec 24, 2019
ci ARROW-7417: [C++] Add a docker-compose entry for CUDA 10.1 Dec 19, 2019
cpp ARROW-7215: [C++][Gandiva] Implement castVARCHAR(numeric_type) functi… Dec 26, 2019
csharp ARROW-7386: [C#] Array offset does not work properly Dec 25, 2019
dev ARROW-7466: [CI][Java] Fix gandiva-jar-osx nightly build failure Dec 24, 2019
docs ARROW-7463 : [Doc] fix a broken link and typo Dec 23, 2019
format ARROW-7463 : [Doc] fix a broken link and typo Dec 23, 2019
go ARROW-7357: [Go] migrate to x/xerrors Dec 9, 2019
integration ARROW-7101: [CI] Refactor docker-compose setup and use it with GitHub… Nov 12, 2019
java ARROW-7437: [Java] ReadChannel#readFully does not set writer index co… Dec 28, 2019
js ARROW-7470: [JS] fix typos Dec 26, 2019
matlab [Release] Update versions for 1.0.0-SNAPSHOT Sep 30, 2019
python ARROW-7420: [C++] Migrate tensor related APIs to Result-returning ver… Dec 24, 2019
r ARROW-7366: [C++][Dataset] Use PartitionSchemeDiscovery in DataSource… Dec 20, 2019
ruby ARROW-7455: [Ruby] Use Arrow::DataType.resolve for all GArrowDataType… Dec 22, 2019
rust ARROW-7460: [Rust] Improve some kernel performance Dec 23, 2019
testing @ 90ae758 ARROW-4219: [Rust] [Parquet] Initial support for arrow reader. Oct 14, 2019
.clang-format ARROW-3313: [R] Move .clang-format to top level. Add r/lint.sh script… Sep 26, 2018
.clang-tidy ARROW-2981: [C++] improve clang-tidy usability Jun 14, 2019
.clang-tidy-ignore ARROW-3313: [R] Move .clang-format to top level. Add r/lint.sh script… Sep 26, 2018
.dir-locals.el ARROW-4930: [C++] Improve find_package() support Nov 5, 2019
.dockerignore ARROW-7146: [R][CI] Various fixes and speedups for the R docker-compo… Dec 6, 2019
.env ARROW-7374: [Dev] [C++] Fix cuda-cpp docker build Dec 16, 2019
.gitattributes ARROW-5488: [R] Workaround when C++ lib not available Jun 12, 2019
.gitignore ARROW-6494: [C++][Dataset] Implement PartitionSchemes Oct 5, 2019
.gitmodules ARROW-4459: [Testing] Add arrow-testing repo as submodule Feb 8, 2019
.hadolint.yaml ARROW-6214: [R] Add R sanitizer docker image Sep 19, 2019
.pre-commit-config.yaml ARROW-4909: [CI] Use hadolint to lint Dockerfiles Mar 18, 2019
.readthedocs.yml ARROW-1142: [C++] Port over compression toolchain and interfaces from… Jun 23, 2017
CHANGELOG.md ARROW-7163: [Doc] Fix double-and typos Nov 13, 2019
CODE_OF_CONDUCT.md ARROW-4006: Add CODE_OF_CONDUCT.md Dec 15, 2018
CONTRIBUTING.md ARROW-6026: [Doc] Add CONTRIBUTING.md Jul 30, 2019
LICENSE.txt ARROW-6341: [Python] Implement low-level bindings for Dataset Dec 13, 2019
Makefile.docker ARROW-6214: [R] Add R sanitizer docker image Sep 19, 2019
NOTICE.txt ARROW-5934: [Python] Bundle arrow's LICENSE with the wheels Jul 15, 2019
README.md ARROW-7101: [CI] Refactor docker-compose setup and use it with GitHub… Nov 12, 2019
appveyor.yml ARROW-7333: [CI][Rust] Remove duplicated nightly job Dec 6, 2019
cmake-format.py ARROW-4363: [CI] [C++] Add CMake format checks Feb 11, 2019
docker-compose.yml ARROW-7417: [C++] Add a docker-compose entry for CUDA 10.1 Dec 19, 2019
header ARROW-259: Use Flatbuffer Field type instead of MaterializedField Aug 18, 2016
run-cmake-format.py ARROW-7169: [C++] Vendor uriparser library Nov 20, 2019

README.md

Apache Arrow

Build Status Coverage Status Fuzzit Status License Twitter Follow

Powering In-Memory Analytics

Apache Arrow is a development platform for in-memory analytics. It contains a set of technologies that enable big data systems to process and move data fast.

Major components of the project include:

Arrow is an Apache Software Foundation project. Learn more at arrow.apache.org.

What's in the Arrow libraries?

The reference Arrow libraries contain a number of distinct software components:

  • Columnar vector and table-like containers (similar to data frames) supporting flat or nested types
  • Fast, language agnostic metadata messaging layer (using Google's Flatbuffers library)
  • Reference-counted off-heap buffer memory management, for zero-copy memory sharing and handling memory-mapped files
  • IO interfaces to local and remote filesystems
  • Self-describing binary wire formats (streaming and batch/file-like) for remote procedure calls (RPC) and interprocess communication (IPC)
  • Integration tests for verifying binary compatibility between the implementations (e.g. sending data from Java to C++)
  • Conversions to and from other in-memory data structures

How to Contribute

Please read our latest project contribution guide.

Getting involved

Even if you do not plan to contribute to Apache Arrow itself or Arrow integrations in other projects, we'd be happy to have you involved:

You can’t perform that action at this time.