Highly Available FTP Solution on Oracle Cloud Infrastructure (OCI)
Goal
The goal of this project was to design and implement a highly available, scalable FTP solution in Oracle Cloud Infrastructure (OCI). The solution aims to provide secure, isolated file storage using ProFTPD for FTP access, with file data stored on a shared NFS server. The setup includes automatic scaling to handle traffic fluctuations and monitoring of the entire infrastructure using Prometheus and Grafana.
Solution
The solution involves the following key components:
- ProFTPD Setup: Two ProFTPD servers were deployed on OCI instances. These servers provide FTP access to clients, allowing them to upload and download files. Each ProFTPD server is configured to authenticate users via virtual users, with directory-level permissions to restrict access to specific directories (i.e., each user can only access their designated directory).
- Network Load Balancer: A Network Load Balancer (NLB) was used to distribute FTP traffic across the two ProFTPD servers, ensuring high availability and fault tolerance. Clients can connect to the NLB IP, which routes traffic to the available ProFTPD servers.
- NFS Server: An NFS server was set up on a separate instance to provide shared storage for the FTP servers. Both ProFTPD instances are connected to the NFS server, allowing them to read and write to the same directory, ensuring data consistency and availability.
- Auto Scaling: Auto Scaling policies were implemented to automatically scale the ProFTPD instances based on load, ensuring that the system can handle traffic spikes and maintain availability during peak demand.
- Prometheus & Grafana Monitoring: A dedicated instance was used to run Prometheus for monitoring the ProFTPD servers, NFS server, and system performance metrics. Grafana was used to visualize the data collected by Prometheus, providing insights into system health and resource utilization.
Impact
- Scalability: The auto-scaling policy ensures that the system automatically adjusts the number of ProFTPD instances based on FTP traffic, maintaining performance during periods of high demand.
- High Availability: The use of a Network Load Balancer for distributing traffic across multiple ProFTPD servers ensures that the FTP service remains available even if one of the ProFTPD instances fails. This design eliminates single points of failure.
- Security and Isolation: By using virtual users in ProFTPD and restricting access to specific directories, users can only access their assigned directories, ensuring that there is no unauthorized access to other users’ files.
- Centralized File Storage: The NFS server allows for centralized, shared storage for both ProFTPD instances, ensuring consistency and high availability of uploaded/downloaded files.
- Operational Efficiency: With Prometheus and Grafana in place, the infrastructure’s health and performance can be monitored in real-time, enabling quick identification of potential issues before they impact the users.
Project Overview
In this project, a highly available and scalable FTP solution was set up on OCI using ProFTPD, NFS, and Network Load Balancer. The system includes two ProFTPD servers for FTP access, an NFS server for shared file storage, and auto-scaling policies to dynamically adjust the system based on load. Virtual users in ProFTPD ensure secure, isolated access to directories for each client, while a Network Load Balancer distributes incoming FTP requests between the ProFTPD servers for high availability. The entire setup is monitored by Prometheus and Grafana for real-time system health and performance tracking. This solution provides a robust, secure, and scalable FTP service with centralized storage, ideal for handling file uploads and downloads at scale.