FileCloud on AWS - User Deployment Guide
Introduction
Use AWS infrastructure (EC2, EBS, S3) to jumpstart your own branded, file storage solution in a few minutes.
Use Cases
- File Sharing Portal- Use FileCloud to create your own own, branded file sharing, sync and mobile access solution for your employees, customers and partners.
- File Sync - Use FileCloud for effortless file synchronization across users computers, smart phones and tablets, so everyone can work together anywhere from any device.
- Client Document Portal - Use FileCloud to create a client document portal on AWS infrastructure to serve your clients, customers and partners.
- Endpoint Backup and DLP - Use FileCloud to securely back up your endpoint computing devices (PCs, Mobile Phones and Servers).
- Enterprise Data Protection and DLP: Use FileCloud's unique data leak prevention (monitor, prevent and fix) capabilities to protect your enterprise data across all your users' devices (computers, mobile phones, tablets).
- File Server Enablement - Use FileCloud's ServerSync to sync/backup your branch office windows files servers to FileCloud running on AWS to get low latency LAN access as well as remote access from anywhere.
Overview of Typical Customer Deployment
FileCloud AMI's are available on the latest versions of Ubuntu and Windows. Depending on your requirements and familiarity, you can choose any operating systems supported by FileCloud. FileCloud stores the file, user and shares metadata in MongoDB (pre-installed in AMI) and actual files can be stored in a disk, AWS Elastic File System or AWS S3. For smaller deployments, disk is sufficient for file storage. For medium and large deployments, our recommendation is to use AWS S3 for file storage. The FileCloud AMI is also preconfigured for document preview and document indexing for full text search. It takes less than 30 minutes to configure the FileCloud AMI and get it running for a production workload.
Prerequisites and Requirements
FileCloud AMIs are completely self contained. You don't need to install any additional software. Basic AWS skills are sufficient to deploy FileCloud on AWS. Simple deployments involve just EC2 and Disk or EC2 and AWS S3. FileCloud AMI's are available as BYOL model. Request a license using this form on our website. Once you get the trial license, upload it to your running EC2 instance. Since FileCloud AMIs are available on Ubuntu and Windows Server OS, you can choose the OS you are familiar with.
Architecture Diagrams
To access FileCloud, you only need port 80 (http) or 443 (https). We strongly recommend you use 443 and only allow SSL access. Depending on the underlying OS, you may need to open up port 22 (SSH access) and 3389 (Remote Desktop) for managing the FileCloud instance. Our recommended security practice is to specify the IP range for SSH and Remote Desktop instead of opening it for access from anywhere.
FileCloud secrets and keys are protected and managed in the FileCloud database. FileCloud supports encryption at rest. To initialize encryption, your administrator may supply an optional master password and start the initialization process. Once the initialization process is started, the following steps occur as part of the process:
- An asymmetric key pair (private/public) known as Master key is generated with the optional master password.
- A symmetric key known as Plain File key is generated.
- The File key created in step 2 is encrypted using the Master private key resulting in an Encrypted File key.
- All the existing unencrypted files (if they exist) in FileCloud storage must be encrypted before the system is ready for use. Look at the next section for more information on file encryption.
If an optional master password was specified, then the administrator should retain the password for future use. Without this password the encryption module cannot encrypt/decrypt files in FileCloud storage.
Additional details on the keys:
Key | Key Details | User Input | Persistence | Remarks |
---|---|---|---|---|
Master public/private key pair |
| Password (optional) | Both private and public keys are persisted. |
|
Plain File Key |
| None | Not persisted |
|
Encrypted File Key |
| None | Encrypted file key is persisted |
|
If you are going to use S3 for Managed File storage, see the security section given below to understand possible file encryption options available.
Planning Guidance
Security
When you deploy FileCloud you can use EBS for managed file storage or you can choose the S3. If you choose S3, use the following instructions to set up your S3 for FileCloud.
Setting up Amazon S3 Credentials
- Log into admin portal.
- Navigate to Settings.
- Select Storage tab.
- Enter the S3 config information. Refer to the following table for more information about each setting.
- Click Save S3 setting.
Field | Description |
---|---|
S3 Key | This is your amazon authentication key (To get your access key, visit Amazon security portal). For IAM user, it requires the IAM Policy for S3 Access given below. |
S3 Secret | This is your amazon authentication secret (To get your access key, visit Amazon security portal). For IAM user, it requires the IAM Policy for S3 Access given below . |
S3 Bucket Name | Provide a bucket name. The bucket should be new (in some circumstance, a previously used bucket in FileCloud could be used). It is very important that the S3 bucket is never modified outside of the FileCloud subsystem. |
S3 Storage Folder | Optional: All files will be stored inside this root storage folder (which is created automatically). |
S3 Region | Optional: Provide the region string. If the region is not provided, then US Standard region will be used. If you are planning to have your bucket in different region(for example, europe or south east) provide the correct region string. The strings should match the region string published by amazon. Note: For govcloud installs, you must use region string: us-gov-west-1 |
S3 End Point URL | Optional: This is the S3 endpoint. Use this if you are planning to use your own S3 endpoint (typically S3 compatible storage) or if it is an unpublished region. If you are using an AWS endpoint, use one of the endpoints published here. |
To fill in the other S3 settings, see Setting up FileCloud Managed S3 Storage.
The Amazon S3 Bucket should NEVER be modified outside of FileCloud subsystem
Do not add/edit/modify files directly using Amazon tools. Doing so will destabilize your FileCloud installation.
IAM Policy for S3 Access
If you are going to use S3 for file storage, FileCloud requires S3 access in order to create bucket and manage it.The IAM user used to manage it must have the following permissions. This shows access to all buckets in your S3 console. You can restrict to specific bucket using the appropriate resource arn. Something like arn:aws:s3:::bucket_name
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:DeleteObject",
"s3:GetObject",
"s3:ListBucket",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::*"
]
}
]
}
Setting up S3 Encryption for FileCloud Managed Storage
S3 Managed Storage Encryption support to protect data at rest is available in Filecloud. The communication between FileCloud to AWS will use SSL encryption resulting in complete protection for data in transit. Once the S3 is set up correctly, a new field "S3 Encryption" will be available under Amazon S3 Storage Settings.
FileCloud supports the following Server Side Encryption:
Encryption Type | Notes |
---|---|
Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3) | All data is encrypted at rest using AES256 bit encryption. The data can only be accessed using the supplied key/secret credentials. The data will be accessible via S3 Console (which should NOT done for FileCloud Managed storage data) |
Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS) | Similar to SSE-S3 but the key itself is managed using Amazon's KMS service. This allows management of specific keys and their permissions for encrypting the data. The data is still encrypted at rest and is accessible via S3 Console with appropriate credentials. |
Server-Side Encryption with Customer-Provided Keys (SSE-C) | This is a new support available from FileCloud v15 on-wards. The data will be encrypted using customer supplied 32 bit encryption key. This option will have SLOWER performance due to restriction on how this data can be decrypted (Amazon server will NOT be able to decrypt the data and the data has be first downloaded to FileCloud server and decrypted). The data will NOT be accessible via S3 console as well. |
To manage S3 encryption,
- Log into Administration Portal
- Navigate to Settings
- Select "Storage" tab
Click on "Manage" button in the S3 Encryption option
Depending on the status of encryption, you will see "Enable encryption" or "Disable encryption" button.
Enabling encryption will attempt to encrypt all available data in the bucket as well as all new data will be encrypted. This can take some time depending on the amount of existing data in the bucket. Modify the encryption setting when there is minimal activity in FileCloud.
Though, changing encryption can be done at any time, we recommend using off-peak hours to avoid any unexpected access issues
For Windows, If your xampp is installed in location other than c:\xampp, then add the following key in <your xampp folder>\htdocs\config\cloudconfig.php
For example, if your xampp is in D:\xampp, then in file D:\xampp\htdocs\config\cloudconfig.php, add the following string (any location before the bottom "?>" line)
define("PHPBIN_PATH","D:\\xampp\\php\\php.exe");
Costs
T2.Medium or T3.Medium is sufficient to run FileCloud for 100 users. File storage cost depends on the storage method you choose (EBS, EFS or S3), the amount of files you will store, and the access pattern.
Sizing
A T3.Medium (unlimited) can handle approximately 30-40 FileCloud calls per second which equates to approximately 100-200 users using FileCloud. Depending on the number of users and their access pattern, you can scale your deployment by choosing a bigger instance or adding more instances.
Deployment Guidance
Deployment Assets
FileCloud pre-built AMI's (Amazon Machine Image) are currently available in Amazon AWS Marketplace for both Linux (Ubuntu) and Windows Server OS.
Steps to Launch FileCloud AMI
- Log in to the AWS management console, and click EC2 (virtual servers in the cloud).
- Click Launch Instance.
- Search for FileCloud AMI in AWS marketplace. Choose the latest version.
The latest version details may be different from the information shown. - Choose the desired Amazon EC2 Instance type. We recommend at least t2.medium or t3.medium. However, for best performance, the "m" series is better. For example m3.medium. If you choose the t2 or t3 series, enable T2/T3 unlimited when you configure the instance.
- Configure the security group to open up port 80/443 for web access.Note: You might need to open other ports such as 443 (HTTPS), depending on your business requirements.
- Complete the Final Review and launch the instance.
- You can see now your FileCloud is running in your AWS dashboard. Note the Public DNS name to access FileCloud.
- Type 'http://<public_dns_name>/ui/admin/index.html' in your browser to access the FileCloud admin portal. If the Webpage fails to load, verify port 80 is open as mentioned above in Step 5.
Default Admin admin Default Password Your amazon instance ID Note Change the admin password upon first login. - Once you have logged into the admin portal, install the FileCloud License.
Request a license using this form on our website to get trial licenses.
- The user name for the underlying Ubuntu OS is 'ubuntu'. Before launching the instance you will be required to create a key pair or you can use your existing key pair.
- If you go to 'http://<public_dns_name>/install' , the page will show all the installed packages in this instance. Check the page and familiarize yourself with FileCloud components. Before going production move the install folder to somewhere else.
- We recommend you to use S3 for file storage instead of the EBS. See the following section (Enabling Amazon S3 Storage) for instructions for setting up S3 for FileCloud file storage.
- The default FileCloud instance uses our SMTP servers and accounts to send emails. Change this to your SMTP server for security purposes.
- Change the admin email to your organization email address. This email address is used in all the emails that sent out from the FileCloud System
Enabling Amazon S3 Storage
Do not change this once the installation is set up and data is already stored. This should only be set up for fresh installs.
When changing the storage type from local to amazons3, the file(s)/folder(s) that have been already stored in the local storage will not be automatically moved to was s3 storage.
In this case, adminstrator has to manually export file(s)/folder(s) from local storage before changing storage type and manually import them after changing storage type.
Be very careful when changing the storage path, If done improperly it could lead to data loss.
To enable Amazon s3 storage as the backend:
- Edit the file WWWROOT/config/cloudconfig.php and change the line
define("TONIDOCLOUD_STORAGE_IMPLEMENTATION", "local");
to read as
define("TONIDOCLOUD_STORAGE_IMPLEMENTATION", "amazons3"); - Rename the file WWWROOT/config/amazons3storageconfig-sample.php to WWWROOT/config/amazons3storageconfig.php
Nothing needs to be added or edited in amazons3storageconfig.php
In Windows WWWROOT is typically c:\xampp\htdocs and in Linux it is /var/www/html
Once you have configured FileCloud storage, follow the site setup instructions to set up the FileCloud site according to your requirements
Operational Guidance
Health Checkup
AWS offers excellent system, instance status checks and CloudWatch monitoring. Pay attention to the CPU utilization, Network In/Out and Network Packet In/Out of your EC2 instance. Using CloudWatch monitoring scripts, you can also monitor memory, swap, and disk space utilization of your EC2 instance.
Apart from the standard AWS monitoring metrics, FileCloud also offers system alerts. FileCloud Alerts are available in FileCloud's Admin portal.
This page tracks all unhandled exceptions and system error messages generated in the FileCloud server. The number of alerts are shown in the Dashboard and the Alerts page shows detailed information about the various errors encountered.
Depending on the error, you might need to take steps to correct the problem. For example, if alerts indicate that system is frequently running out of memory, then system memory may need to be increased.
To view alerts:
- Log into the Administration portal.
- On the left navigation panel, click Alerts.
The following screenshot shows errors detected by FileCloud. The alerts are categorized as Informational, Warning, Critical and Fatal. Always pay attention to critical and fatal errors. FileCloud administrators also receive periodic Administrator Summary emails that show the number of alerts.
Backup and Recovery
FileCloud supports unlimited file versioning and a recycle bin. You can configure these options by logging in to the FileCloud admin portal. These features provide protection from accidental deletions by users.
In addition, take periodic snapshots of your running instance for disaster recovery and as an additional backup for the FileCloud database and app. If you are not taking snapshots of your running instance, at minimum, make sure you are backing up the mongodb database to a disk or S3 using AWS CLI. If you have the FileCloud database, you can recover the FileCloud application from instance/service failure.
The following command backs up mongodb to a designated s3 bucket (you can also make it a cron job so that it runs periodically).
cd /var/lib/mongodb aws s3 sync . s3://my-bucket/fileclouddbbackup/
In case of instance failure, start a new FileCloud AMI, and follow the instructions below to bring the instance up and running.
1. Before making any changes, stop the mongodb service.
service mongod stop
2. Copy the backup database files back to /var/lib/mongodb from the s3 bucket.
cd /var/lib/mongodb aws s3 sync s3://my-bucket/fileclouddbbackup/ .
3. Start the mongodb service using following command.
service mongod start
Routine Maintenance
Routine maintenance helps you keep your FileCloud system updated to the latest version.
Generally, you can find notifications of new FileCloud release availability by:
- Subscribing to the FileCloud Mailing List.
Seeing the version update available in the FileCloud Admin Dashboard.
Emergency Maintenance
If the EC2 instance where you are running FileCloud is degraded, You have two options:
- Take a snapshot of the EBS disk and start an instance from the snapshot.
or - If option 1 is not feasible, start a new FileCloud AMI and copy the backed up FileCloud Database files to /var/lib/mongodb (Linux) or c:\xampp\mongodb\bin\data (Windows). Then start the mongod service.
Reference Materials
FileCloud Site Setup Guide - FileCloud Site Setup Guide
FileCloud End User Guide - FileCloud End User Guides
FileCloud API Getting Started Guide - FileCloud Developer Guide