My Experience with Storage Made Easy
With Box finally shuttering it’s long-depreciated WebDAV portal, I went from one edge case to another by using Storage Made Easy to access my music in Box without moving it.
For a bit of background, SME is an enterprise-oriented solution for working with and managing multiple clouds. Many years ago, I purchased lifetime access, which I don’t believe they offer anymore. Any clouds you connect to the server appear as a folder in SME’s root directory, and some plans (like mine) offer WebDAV and FTP access to the whole thing. In fact, Astiga includes a preset to add the url for the service from back before the change of management.
I first tried adding a Box account using SME’s WebDAV. It added successfully, but the file scanner just kept repeating a GET on my starting folder no matter how I set it up. I tried again with sftp and that seemed to work better with the scanner successfully navigating through everything. However, I’m noticing that occasionally one of the tracks (around 1 in 5-10) will fail to download. SME’s documentation mentions a maximum of 5 concurrent connections, so I’m not sure if Astiga might be running up against that or if it’s something else in this convoluted setup. I figure an additional sync will probably fix everything, but my legacy 50GB box account is full so I’m in for a long wait. Also interested to see what performance is like.
Comments
I can see some of these errors - I'll take a look later to see if we can add more logging to work out the reason the file isn't downloaded - currently it isn't shown fully.
Thanks for the write-up - an interesting solution!
I've added logging - I noticed a sync has already started for your SFTP account so I'll check back again later. I think it might take some more logging and tracing yet though.
Do you have a link to read about the five concurrent connections limitation?
Found it here: https://eu.storagemadeeasy.com/faq/
Is there a limit to the number of connections I can use for CloudFTP?
Yes, you are limited to 5 connections when using CloudFTP. This means we allocate connections for one session.
Oh - because the files are all added now (it looks like you performed another sync later which, this time, picked up the files?) this sync didn't show the issue - as it didn't attempt to add the files.
Would it be ok to run a force rescan? In case you have other limitations I didn't want to kick that off.
I actually have another account on Box, so let me try adding that one and see what happens.
Edit: I added my second account and I’m getting the same errors. I also turned off auto sync on both accounts, so I’ll run the next sync on my second account after you look it it. There’s a setting on SME’s side that has it ping the cloud as you browse it to make sure the directory contents are up-to-date. This setting is on right now, but turning it off might give the scanner a faster wait time.
Yep, I took a look, thanks.
As I suspect, there's little extra info, but it does confirm where the error is coming from (the bit of code that performs the fetch of the file).
I added some more logging, can you check again?
I just tried to do a full resync of my second box account, but it sent me to a page that said “Slim application error”. I tried this a couple times with the same result. I can’t tell if the sync job started successfully or not.
Edit: Also, trying to edit metadata on my first box account isn’t letting me browse the folder for some reason. I click on the arrow, it thinks for a moment, then the arrow’s gone. Yet I’m able to browse the second box account with the metadata editor. Weeeeird…
Should be good to sync again now?
Started a full resync on one account.
Oh s*#$t! Sorry, I was still thinking of WebDAV so I had edited the logging there - not SFTP. Can you run another scan? I'm sorry this is taking so long, but it's kind of in the nature of this type of thing...
Ok, it’s running.
Thanks - I'll have to check Monday morning now.
Ok. I might try a test with SME’s WebDAV as well at some point to see if anything helpful gets logged.
Sorry, I was only just checking this. It turns out my logging was bad again. Could you run another full refresh? Thanks so much.
Okay, another full refresh is running.
Would it be ok if I kicked these syncs off for you, so long as I just run the sync for this one account?
Yes, that’s fine. I also need to figure out why my other box account started showing no contents the last time I tried syncing it… Honestly if and when native box support happens I will probably just use that. This setup is a little too overly complicated.
Thanks. Just trying to run this and it looks like the folder is showing as empty. Are you seeing the same thing?
I noticed that on my larger account after we started to work on my smaller account. I thought it might have been some sort of response to the automatic resync, which was enabled at the time, since the files are still there and accessible from Box and SME. But if Astiga was being blocked somehow, I’d think that it wouldn’t let me edit the account because it wouldn’t detect a valid connection anymore.
So yeah, not sure what else to try at this point. Maybe SME isn’t the answer.
Looks like it's still happening... It would help if a representative error message was sent! I guess we could just wait and come back to it.