This article was written quite some time ago and might not be relevant anymore. Please be aware of this before extracting and/or applying anything from this article.

Please take in mind that this article was written in February back in 2016. It might still be a valid solution to the problem I was facing while synchronising with Transip's Stack Storage and my home server, I feel like there should already be a better solution. I might research this myself as I will be performing another big sync very soon.

As a fairly happy TransIP customer/user I've finally decided to transfer my personal data towards their latest service named STACK. This turned out to be quit more difficult than I had expected, but with some digging around (and some support from their side) I've eventually found the "solid" setup to properly transfer all of my data safely.

As we all know, the storage for STACK is TransIP's old hardware, implemented with a custom OwnCloud setup. The OwnCloud manual suggests installing their synchronization tools. This however did not work for me, as my data is installed on a Debian server, which I almost solely use over SSH.
The next solution would be mounting STACK over webdav (using davfs2). This would just leave you to choose whether you'd like to copy or rsync your data to the mount and be done with it. Unfortunately this did not work for me either, as I was transferring around 500GB of data and the system didn't provide enough resources for this. Strange errors like "resource is full, no more space left on device" where occurring, even when STACK was mounted on a 3TB disc with atleast 50% free disc space left.
After contacting TransIP's support desk they provided me with a few manual pages from OwnCloud, where I found the OwnCloudcmd tools.

Now, what is owncloudcmd? Owncloudcmd basically has all the logic and routines for synchronizing your data and is used by your owncloud GUI but can also be run manually from the commandline. This sounds like what we want!

But when transferring big files it may sometimes fail as it will slice your big files into smaller bits, called chunks. Uploading these chunks can sometimes cause a timeout, meaning your owncloudcmd client will abort the upload. Luckily it also stores some (meta)data inside your local owncloud folder so it knows where it stopped and resuming the process would not be a big trouble. The owncloudcmd tool does not provide a form of automated synchronization, like with the usage of a owncloudcmd daemon or inotifywatcher, we must run this command each given time so we can check for file differences between the client and server versions of our owncloud folder and synchronize any differences.

This doesn't sound entirely waterproof but luckily we can automate all this with the use of supervisord! Wanna know how?

First, install the following packages on your debian / ubuntu system (other distro's might work differently with packagenames but the setup is quit the same):

$ sudo apt-get update
$ sudo apt-get install owncloud-client supervisor

Now we have owncloud-client package and supervisor installed on our system. The owncloud-client package contains the owncloudcmd tool and supervisor is a process monitoring tool which will make sure our process is active the entire time we want it too.

Next, let's first take a look at how to use the owncloudcmd command:

$ owncloudcmd [OPTIONS...] sourcedir owncloudurl

By default, owncloudcmd reads the client configuration and uses the credentials of the GUI synchronization client. If no client is configured (which, in our case, is not configured), or if you choose to use a different user to synchronize, you can specify the user password setting with the usual URL pattern. In our case this would be:

$ owncloudcmd -s /path/to/owncloud/folder https://YOURUSERNAME:YOURPASSWORD@YOURUSERNAME.stackstorage.com/remote.php/webdav

Noticed the -s? Well, by default the owncloudcmd outputs a pretty verbose log to the terminal. The -s parameter, which actually is a alias for --silent, tries to make it a little less verbose.

Great, we now have our commandline command figured out. Next thing is to place this inside our supervisor configuration. Open a terminal and create the following file at /etc/supervisor/conf.d/owncloudcmd_sync.conf:

command=/usr/bin/owncloudcmd -s /path/to/owncloud/folder https://YOURUSERNAME:YOURPASSWORD@YOURUSERNAME.stackstorage.com/remote.php/webdav/
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisord/owncloudcmd_sync.stderr.log

The configuration begins with declaring a name for the program to run, which is owncloudcmd_sync. Next we actually pasted our command from earlier, with the extension that the complete path towards the owncloudcmd executable is used. This is necessary!
We then say that the program should be automatically started as well as restarted when the process is finished or aborted duo to errors. We only declare one logfile as the owncloudcmd just sents output using stderr instead of stdout.

Now that we have our supervisord configuration all set up we want to begin to use it. We just have to tell supervisord to look for new/updated program configurations first. This is done by executing the following:

$ sudo supervisorctl reread

and

$ sudo supervisorctl update

As we configured that supervisord should autostart our owncloudcmd command it should already be running at this point. To check this you could tail the logfile we just configured or use supervisorctl to listen in on the command's environment.

I hope this setup helped you with configuring your owncloud synchronization as it did with mine. If you have any questions or suggestions feel free to leave them in the comments below!

This article was written quite some time ago and might not be relevant anymore. Please be aware of this before extracting and/or applying anything from this article.