Awesome Data Analysis
Section: Tools · A high-performance table format for huge analytic datasets.
FreshScore 80
Entry
Apache Iceberg
Appears in 4 awesome lists
Iceberg is an ACID-compliant, high-performance format built for huge analytic tables (containing tens of petabytes of data), and it brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work…
Open github.comapache/iceberg
Found in these lists
Awesome Open Source AI
Section: 1. Core Frameworks & Libraries · High-performance open table format for huge analytic tables. Brings SQL table reliability to big data with time travel, hidden partitioning, and schema evolution. Works with Spark, Trino, Flink, Presto, Hive and Impala. Apache 2.0 licensed.
FreshScore 89
Awesome Production Machine Learning
Section: Data Storage Optimisation · Iceberg is an ACID-compliant, high-performance format built for huge analytic tables (containing tens of petabytes of data), and it brings the reliability and simplicity of SQL tables to big data, while making it possible for engines like Spark, Trino, Flink, Presto, Hive and Impala to safely work…
FreshScore 92
Awesome Spark
Section: Storage · Upserts, Deletes And Incremental Processing on Big Data..
SlowScore 66
Often listed next to it
TensorFlow
How to use the Hexagon Delegate to speed up model inference on mobile and edge devices. Also see blog post Accelerating TensorFlow Lite on Qualcomm Hexagon DSPs.
In 23 listsDetails
PyTorch
(label: good first issue) PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing.
In 16 listsDetails
Luigi
Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
In 15 listsDetails
transformers
(formerly known as pytorch-transformers and pytorch-pretrained-bert) provides state-of-the-art general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet, CTRL...) for Natural Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in…
In 14 listsDetails
Milvus
Milvus is a cloud-native, open-source vector database built to manage embedding vectors generated by machine learning models and neural networks.
In 14 listsDetails
Apache Airflow
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
In 13 listsDetails