DOCS
  • Saagie Documentation
    latest
      • Introduction
    • What’s New?
      • 2023
      • 2022
      • 2021
      • 2020
    • Getting Started Guide
      • Getting Familiar with the Interface
      • Managing Your User Profile
      • Starting With the Saagie Project Example
    • Data Team Guide
      • Projects Module
        • About the Projects Module
        • Projects
          • Managing Projects
          • Project Settings
        • Jobs
          • About Jobs
          • Managing Jobs
          • Job Settings
        • Pipelines
          • About Pipelines
          • About Conditions in Pipelines
          • Managing Pipelines
          • Pipeline Settings
        • Apps
          • About Apps
          • Managing Apps
          • App Settings
        • Storage
          • About Storage
        • Managing Environment Variables
        • Managing Docker Credentials
      • Monitoring Module
        • About the Monitoring Module
      • Operations Module
        • About the Operations Module
      • Catalog Module
        • About the Technology Catalog
        • Repositories
          • About Repositories
          • Managing Repositories
          • Repository Settings
        • Technologies of the Official Saagie Repository
      • Security Module
        • About the Security Module
        • Managing User Accounts
        • Managing Groups
      • Add-On
        • Saagie Usage Monitoring
          • About Saagie Usage Monitoring
          • Saagie Usage Monitoring Default Dashboards
          • Customizing Your App
    • Developer Guide
      • Software Development Kit (SDK)
        • Creating and Managing Technologies
          • Creating the Metadata Files and the Zip Archive
          • Creating a Repository
          • References
        • About Docker Images Within Saagie
      • Application Programming Interface (API)
        • Saagie GraphQL API
        • Saagie Python API
    • Administrator Guide
      • About Saagie Architecture
        • How Does Saagie Work?
        • Interactions Between Saagie and External Components
        • Prometheus
        • Conf’o’rama
      • Installing and Setting Up Saagie
        • System Requirements
        • Creating or Configuring Your Kubernetes Cluster
          • Using Amazon Elastic Kubernetes Service (EKS)
          • Using Microsoft Azure Kubernetes Service (AKS)
          • Using Google Kubernetes Engine (GKE)
          • Using Another Service Platform
        • Downloading and Configuring Saagie
        • Running the Saagie Installer
        • Creating a Domain Name System (DNS) Entry
        • Deploying and Updating Your SSL Certificate
        • Enabling LDAP Authentication
        • Upgrading Saagie
      • Maintaining Your Platform
        • Starting and Stopping Saagie
        • Adding a Node in Your Saagie Cluster
        • Changing the Data Lake URL in Saagie
        • Monitoring Technologies
        • Monitoring Logs
          • Audit Logs
          • Operational Logs
    • How-To Guides
      • Apache Sqoop
        • Import Data From a MySQL, Oracle, PostgreSQL, or SQL Server Database
        • Import Data From Other Relational Database Management System (RDBMS)
      • Apache Spark
        • Configure Spark Resources
        • Read and Write Files or Tables With PySpark
          • Read and Write Files From Amazon S3 Bucket With PySpark
          • Read and Write Files From HDFS With PySpark
          • Read and Write Tables From Hive With PySpark
        • Read and Write Files or Tables With Spark Scala
          • Read and Write Files From HDFS With Spark Scala
          • Read and Write Tables From Hive With Spark Scala
        • Package Your Spark Scala Code With the Assembly Plugin
        • Integrate Spark Streaming With Kafka
      • HDFS
        • Manually Upload Large Files to HDFS
        • Read and Write Files From HDFS, WebHDFS, and HTTPFS With HDFS
      • Java/Scala
        • Read and Write Files or Tables With Java/Scala
          • Read and Write Files From HDFS With Java/Scala
          • Read and Write Tables From Hive With Java/Scala
          • Read and Write Tables From Impala With Java/Scala
          • Read and Write Files From MongoDB With Java/Scala
        • Use Java/Scala on Impala With High Availability
      • Kerberos
        • Adapt Your Job to Connect to Kerberos
      • Notebooks
        • Create RStudio User Accounts
        • Configure Git on RStudio
      • Python
        • Read and Write Files or Tables With Python
          • Read and Write Files From Amazon S3 Bucket With Python
          • Read and Write Files From Elasticsearch With Python
          • Read and Write Files From HDFS With Python
          • Read and Write Tables From Hive With Python
          • Read and Write Tables From Impala With Python
          • Read and Write Files From MongoDB With Python
          • Read and Write Tables From MySQL With Python
          • Read and Write Tables From PostgreSQL With Python
        • Use Python on HDFS or Impala With High Availability
      • R
        • Use R With a Proxy
        • Update R Packages
        • Use External R Packages
        • Create Hive Dynamic Tables
        • Read and Write Files or Tables With R
          • Read and Write Files From Amazon S3 Bucket With R
          • Read and Write Files From Elasticsearch With R
          • Read and Write Files From HDFS With R
          • Read and Write Tables From Hive With R
          • Read and Write Tables From Impala With R
          • Read and Write Files From MongoDB With R
        • Use R on HDFS With High Availability
        • Use R on Impala With High Availability
      • Talend
        • Read and Write Files or Tables With Talend
          • Read and Write Files From Amazon S3 Bucket With Talend
          • Read, Write, and List Files From HDFS With Talend
          • Read and Write Tables From Hive With Talend
          • Read and Write Tables From Impala With Talend
          • Read Files From MongoDB With Talend
        • Copy a File From HDFS to the Local Computer
        • Produce and Consume a Message in the Kafka Cluster
        • Use Talend on HDFS With High Availability
    • FAQ
      • What are the basic Kubernetes commands I need to know to use Saagie?
      • How do I connect to a container?
      • How can I see the component logs?
      • How can I increase the storage size of a Persistent Volume Claim (PVC)?
      • How do I restore a component?
      • How can I contact Saagie support?
      • How do I request a feature?
      • Glossary
      • Site Map
  • How-To Guides
  • HDFS

HDFS

This section describes how to use HDFS with Saagie.

In this section:

  • Manually Upload Large Files to HDFS

  • Read and Write Files From HDFS, WebHDFS, and HTTPFS With HDFS

Integrate Spark Streaming With Kafka Manually Upload Large Files to HDFS
Contact Support

This page was built using the Antora default UI.

The source code for this UI is licensed under the terms of the MPL-2.0 license.

×