Provisioning & Configuration
Configuring on deployment
The transfer-agent supports resource provisioning via configuration file. This config can be passed by direct path, or by filename.
masv-agent -config=config.yml
If passed by filename, the agent will check the config directory:
/config
for the docker image release~/.masvsrv
for the binary release.
auth:
api_key: your-api-key
# email: test@test.com
# password: topsecret
​settings:
upload_rate_limit:
enabled: true
rate_limit_bps: 10000000
schedule_enabled: true
schedule:
fri:
enabled: false
start: "09:00"
end: "17:00"
mon:
enabled: true
start: "09:00"
end: "17:00"
sat:
enabled: true
start: "09:00"
end: "17:00"
sun:
enabled: true
start: "09:00"
end: "17:00"
thu:
enabled: true
start: "09:00"
end: "17:00"
tue:
enabled: false
estart: "09:00"
end: "17:00"
wed:
enabled: false
start: "09:00"
end: "17:00"
download_rate_limit:
enabled: true
rate_limit_bps: 20000000
schedule_enabled: true
schedule:
mon:
enabled: true
start: "09:00"
end: "17:00"
fifo_mode:
enabled: true
active_download_count: 1
active_upload_count: 1
disk_write_mode:
calibration_mode: 0
manual_write_modes:
2049: 0
2050: 1
portal_download_automations:
- 1:
name: portal-download-auto
portal_subdomain: YourPortalSubdomain
destination_folder: /data/portal-downloads
create_package_folder: true
effective_time: 2021-07-30T00:00:00Z
enabled: true
priority: 0
portal_upload_automations:
- 1:
name: portal-upload-auto
portal_subdomain: YourPortalSubdomain
portal_password: PortalPassword
path: /data/watch
sender_email: youremail@yourdomain.com
timeout: 5
delete_files_after_upload: false
send_loose_files: false
package_name_suffix: portal-suffix
message: portal upload msg
blacklist: ["*.png"]
enabled: true
priority: -1
team_upload_automations:
- 1:
name: team-upload-auto
team_id: TEAM_ID
path: /data/watch2
timeout: 5
recipient_emails: [recipient@example.com]
delete_files_after_upload: false
send_loose_files: false
package_name_suffix: PACKAGE_SUFFIX
message: team upload msg
download_limit: 7
delete_after: 3
download_password: DownloadPassword
blacklist: ["*.txt"]
enabled: false
priority: 1
unlimited_storage: false
tag:
id: tag_id
name: tag_name
Auth
The authentication method for the MASV API. Depending on the mechanism used, it can either have:
- An api_key property for API key based authentication (recommended)
- email and password properties for user-based authentication (not recommended)
Settings
The following settings can be modified via the provisioning config file:
- upload and download rate limit schedules
- fifo mode settings
- disk write mode (serial vs parallel)
Rate Limit Schedules
The rate limit schedules allow the app to automatically limit the transfer bandwidth during certain times and days of the week. The format for start and end is a HH:MM
time stamp related to the local time. Days can be omitted, which will result in that day being disabled in the schedule.
FIFO Mode
FIFO mode settings, when enabled, determine the number of concurrent uploads/downloads that can be allocated bandwidth.
Disk Write Mode
Disk Write Mode settings determine whether the app will run in the standard parallel write mode, or serial write mode. In serial write mode only one write operation will be allowed at a time per disk.
If using the manual calibration mode (0), then the per disk write modes should be specified, keyed by the disk ID. For the per disk write modes, 0 indicates parallel, and 1 indicates serial.
The ID's for your disks can be found using the /api/v1/settings/disk_write_modes
endpoint, and will be the keys in the response disk_write_modes
objects.
{
"calibration_mode": 0,
"disk_write_modes": {
"2049": {
"disk_label": "/dev/sda1",
"requires_automatic_write_mode_calibration": true,
"write_mode_automatic": 0,
"write_mode_manual": 1
},
"2050": {
"disk_label": "/dev/sda2",
"requires_automatic_write_mode_calibration": true,
"write_mode_automatic": 0,
"write_mode_manual": 1
}
}
}
Portal Download Automations
Portal download automations will automatically poll the specified portal, and download all finalized packages that it finds that were created after the effective_time
field.
Upload Automations
Portal and team upload automations both watch the specified parent folder for changes to the children. If no changes are detected in a child folder (or file, if send_loose_files=true
) after timeout
minutes, then the folder/file is uploaded to the portal/recipients.