Statements (74)
| Predicate | Object |
|---|---|
| gptkbp:instance_of |
gptkb:park
|
| gptkbp:bfsLayer |
4
|
| gptkbp:bfsParent |
gptkb:Matei_Zaharia
|
| gptkbp:category |
gptkb:software_framework
gptkb:Big_Data Data Processing Stream Processing |
| gptkbp:dependency |
gptkb:Scala_2.12
Java 8+ Hadoop 3.2+ |
| gptkbp:developer |
gptkb:software_framework
|
| gptkbp:features |
Improved performance
Support for Kubernetes Better integration with Delta Lake Enhanced machine learning library Improved UI New SQL functions Support for Python 3.10 |
| https://www.w3.org/2000/01/rdf-schema#label |
Spark 3.8
|
| gptkbp:language |
gptkb:Java
gptkb:R gptkb:Library gptkb:Skrull |
| gptkbp:license |
Apache License 2.0
|
| gptkbp:notable_feature |
Enhanced security features
Improved error messages Performance optimizations Improved resource management Support for data warehousing Improved documentation Support for user-defined functions Support for complex data types Support for data analytics Support for new file formats Support for multi-tenancy Support for data archiving Support for data lineage Support for data quality checks Support for data transformation Support for machine learning pipelines Support for real-time analytics Support for data sharing Improved caching mechanisms Support for new data sources Improved support for data governance Improved support for data science Improved support for data visualization Support for data compliance Support for data engineering Adaptive Query Execution Columnar Data Processing Dynamic Partition Pruning Improved Data Frame API Improved integration with BI tools Improved performance for joins Improved support for data collaboration Improved support for data exploration Improved support for data integration Improved support for data lakes Improved support for data management Improved support for data migration Improved support for data standards Improved support for graph processing Improved support for window functions New M Llib algorithms Support for Arrow Support for SQL on streaming data Support for batch and streaming workloads Support for custom aggregations Support for data governance frameworks |
| gptkbp:predecessor |
Spark 3.6
|
| gptkbp:release_date |
2023-08-01
|
| gptkbp:successor |
Spark 3.7
|
| gptkbp:website |
https://spark.apache.org/
|