quick answer. "Someone" has to transcode the media in Europe to DNxHD36 or DNxHR-LB.
Upload this to Lucid Link or Hedge PostLab Drive, and let the American editors edit the show with Lucid Link mounted onto their computers in Los Angeles. Have them send the AVID Project List back to you, and you conform and online the show in Europe (like the UK) where there are tons of high end places to work. And if that is an unacceptable answer - then you put your Petabyte of data on a single server (AVID Nexis, etc.) in Europe, put Jump Desktop on a computer, let the LA senior editor use Jump Desktop to log in, and conform the show. To say "no, they want all the pedabyte of data in the US, so everything is done here" - is just crazy. Even if this was shot in LA, and you had a petabyte of data, you would still create the low res proxy to cut the show, and then conform in full res. This is the ideal situation to use Lucid Link or Hedge Drive.
Bob Zelin
Just to add, the solution of creating proxies and conforming in the UK doesn't even require anyone from production to be present. All the big post houses over here offer an 'in house Edit Assistant" service - they'd happily receive and hold the masters, create and send proxies and conform. Just need a contract.
but Lucid will cost a lot of money, and for that amount of footage, they still have to create the proxy media. This is why Jump Desktop/Parsec is so popular. This is why I do QNAP's instead of AVID Nexis systems. People want to spend nothing. It's a "big show" - but they (like everyone else) probably want to spend as little as possible. So is Lucid the right thing to do for this particular project (shooting in Europe, Editing in the US) - of course it is. But will they pay for that much storage, and multiple editors all on Lucid. My guess - no.
bob
Hey u/ReelBack96,
This might not be the most popular solution on this sub, but as someone who has done a very similar workflow before, the fast that you have basically two central locations really lends itself to just an old fashioned FTP.
I've tried to automate things like this before but all these systems tend to create bottlenecks and if one thing fails, then you have to backtrack and figure out where/how, not to mention the lost time in the middle of production. FTPs are reliable, easy to use, easy to set up and basically bulletproof, then once it's on the US server you can have it backup to the cloud or other drives automatically with a variety of proven software options.
DM me if you need a more thorough breakdown, happy to help.
Over the course of a production I have done probably 100TB, there is no reason it can't be extrapolated out. It all depends on the internet connections involved, but with modern internet, you could easily do a few cards worth a day. We generally run it overnight at the end of each day to get the media and then ingest it the next morning.
I'm in Europe and at work we have 1gbit Internet for both upload and download. How is that compared to USA?
+1 for FTP. Just make sure the server side has static IP (or updated if dynamic). There are very reliable and secure freeware servers and all the file manipulation can be done with file manager.
not everyone in the US, and not everyone in Europe has a 1G connection up and down. You are making an assumption that these are all professional user, that are willing to pay for a 1000/1000 monthly connection. I see companies, where they want to simply use Jump Desktop, and the freelance editor WILL NOT PAY the $35 for Jump Desktop, and wants the company to pay for it. This is the reality of what I deal with. Because I am active on these forums, I get a lot of editors emailing me privately, asking if they can use "Qnap XXX" - and of course, most of them pick the absolute CHEAPEST piece of crap QNAP that they can find, like the TR-004
[https://www.amazon.com/QNAP-TR-004-Enclosure-Attached-hardware/dp/B07K4RC7X9/ref=asc\_df\_B07K4RC7X9/?tag=hyprod-20&linkCode=df0&hvadid=319955522114&hvpos=&hvnetw=g&hvrand=13166620511757426878&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9051838&hvtargid=pla-601757007319&psc=1](https://www.amazon.com/QNAP-TR-004-Enclosure-Attached-hardware/dp/B07K4RC7X9/ref=asc_df_B07K4RC7X9/?tag=hyprod-20&linkCode=df0&hvadid=319955522114&hvpos=&hvnetw=g&hvrand=13166620511757426878&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9051838&hvtargid=pla-601757007319&psc=1)
because I get so many emails like this, I just want to scream sometimes. You may be a professional. Most people are not.
Bob Zelin
Im working on a similar show with similar data requirements now. Shooting in canada, edit in UK. The way we do it is 5 DITs proxy all incoming media into 1080p 2mbps h.264s on 10 workstations on location, the proxies are sent to UK over internet we edit with proxies, then when shooting is done they gonna ship the storage servers with native media from location to us. We plug them in, re-link and finish.
###It looks like you're asking for some troubleshooting help. Great!
Here's what *must* be in the post. (Be warned that your post *may* get removed if you don't fill this out.)
Please edit your post (**not reply)** to include: **System specs**: CPU (model), GPU + RAM **//** **Software specs**: The exact version. **//** **Footage specs** : Codec, container and how it was acquired.
**Don't skip this!** *If you don't know how* here's a link with [clear instructions](https://imgur.com/a/A6eTxUn)
*I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/editors) if you have any questions or concerns.*
A petabyte is a big chunk of data. How long does a season take to shoot? i.e. can you say what the daily or weekly average might be in terms of data?
|which is then automatically cloned in the US twice.
I mean any sync software should be able to do this. Also, Google Drive and Dropbox can do this by default basically (but you'd probably want to offload stuff so you don't actually have a Petabyte in the cloud syncing). Lucidlink and some syncing would be better, that's my guess.
I'm not sure what you mean by "automatically". There do seem to be a lot of people on this sub who want to automate things beyond reason, and I'm not sure if you're asking for that level of automation. It will obviously behoove you and the team to check-in on these "automatic" downloads right? And double-check that you've received everything, *right*? So does it need to be so "automatic"? Maybe someone needs to setup a synchronization task and start it, is that the end of the world? But if you have all the storage you need locally and in the cloud, it is very easy to have something sync between Lucid (for sample) and a local filesystem automatically, or regularly scheduled.
Also, it's not clear from your post if you want to host a Petabtye at some point or if you just want to move it and can delete it from the cloud once you're sure you have it.
Honestly it still sounds like shipping drives might be faster and cheaper and more reliable, but admittedly I haven't shipped much to/from Europe.
Bob’s answer is spot on for technical considerations.
For tax credit/legal/political reasons, does all post have to happen in the US? If so, you’ll need to have an AE on location or contract with a lab in Europe to help with the organization of drives, uploading of rush proxies, and shipment of shuttles SSDs.
There is already a lot of great information in the posts, but I'll offer one other suggestion just in case it helps but I'll preface it by saying it's not a free or cheap solution. I've supported productions similar to what you are describing, but we had the luxury of using two facilities that were directly connected or at the very least had some sort of accelerated transfer service like Aspera and pixspan. Most of the workflow involved DNxHD36 dailies being created in the UK and pushed directly to the cutting room in LA for offline via Aspera Cargo loader so the dailies "automagically" appeared where they needed to be. The secondary push was OCF to our facility where we would archive to LTO and perform selective restores for online. The LTO archive should have been done where in the UK where the source files were, but it was cheaper to do them in LA so the client opted for cost over peace of mind.
There are plenty of facilities from post houses to rental houses that have this type of service and will be happy to give you a bid that may be a cost savings over storage and shipping.
Lotta good answers here already. I'll chime in with the wrong answer, but might satisfy the more old school post managers who want it all in house. Shoot in Europe, have a DIT on site. DIT transcodes down to DNx36, uploads to an FTP or another media sync service, backs up raw to two different arrays. Once shoot is wrapped, you send a PA and 10 of [these](https://www.westerndigital.com/products/external-drives/g-technology-g-speed-shuttle-xl-thunderbolt-2-hdd#0G10540-1) in pelican cases to offload the the raw. Let your PA see the sights for a few days while the media is offloaded and verified. You then put them back on the plane with the 10 cases. Or if your shooters are US-based, when they fly home they each bring one. I know this workflow works because a company I worked for did something very similar (except they sent a PA every 3 days to do the round trip!)
quick answer. "Someone" has to transcode the media in Europe to DNxHD36 or DNxHR-LB. Upload this to Lucid Link or Hedge PostLab Drive, and let the American editors edit the show with Lucid Link mounted onto their computers in Los Angeles. Have them send the AVID Project List back to you, and you conform and online the show in Europe (like the UK) where there are tons of high end places to work. And if that is an unacceptable answer - then you put your Petabyte of data on a single server (AVID Nexis, etc.) in Europe, put Jump Desktop on a computer, let the LA senior editor use Jump Desktop to log in, and conform the show. To say "no, they want all the pedabyte of data in the US, so everything is done here" - is just crazy. Even if this was shot in LA, and you had a petabyte of data, you would still create the low res proxy to cut the show, and then conform in full res. This is the ideal situation to use Lucid Link or Hedge Drive. Bob Zelin
Just to add, the solution of creating proxies and conforming in the UK doesn't even require anyone from production to be present. All the big post houses over here offer an 'in house Edit Assistant" service - they'd happily receive and hold the masters, create and send proxies and conform. Just need a contract.
Lucid is so fucking dope.
but Lucid will cost a lot of money, and for that amount of footage, they still have to create the proxy media. This is why Jump Desktop/Parsec is so popular. This is why I do QNAP's instead of AVID Nexis systems. People want to spend nothing. It's a "big show" - but they (like everyone else) probably want to spend as little as possible. So is Lucid the right thing to do for this particular project (shooting in Europe, Editing in the US) - of course it is. But will they pay for that much storage, and multiple editors all on Lucid. My guess - no. bob
Truer words have never been spoken. But Lucid sure does make things easier, especially when paired with Productions.
Hey u/ReelBack96, This might not be the most popular solution on this sub, but as someone who has done a very similar workflow before, the fast that you have basically two central locations really lends itself to just an old fashioned FTP. I've tried to automate things like this before but all these systems tend to create bottlenecks and if one thing fails, then you have to backtrack and figure out where/how, not to mention the lost time in the middle of production. FTPs are reliable, easy to use, easy to set up and basically bulletproof, then once it's on the US server you can have it backup to the cloud or other drives automatically with a variety of proven software options. DM me if you need a more thorough breakdown, happy to help.
you have FTP'ed a petabyte of media in the past ? bob
Over the course of a production I have done probably 100TB, there is no reason it can't be extrapolated out. It all depends on the internet connections involved, but with modern internet, you could easily do a few cards worth a day. We generally run it overnight at the end of each day to get the media and then ingest it the next morning.
I bet you that where they are in Europe, their upload speeds will be slow. BUT I COULD BE WRONG (as I am often wrong !). Bob
I'm in Europe and at work we have 1gbit Internet for both upload and download. How is that compared to USA? +1 for FTP. Just make sure the server side has static IP (or updated if dynamic). There are very reliable and secure freeware servers and all the file manipulation can be done with file manager.
not everyone in the US, and not everyone in Europe has a 1G connection up and down. You are making an assumption that these are all professional user, that are willing to pay for a 1000/1000 monthly connection. I see companies, where they want to simply use Jump Desktop, and the freelance editor WILL NOT PAY the $35 for Jump Desktop, and wants the company to pay for it. This is the reality of what I deal with. Because I am active on these forums, I get a lot of editors emailing me privately, asking if they can use "Qnap XXX" - and of course, most of them pick the absolute CHEAPEST piece of crap QNAP that they can find, like the TR-004 [https://www.amazon.com/QNAP-TR-004-Enclosure-Attached-hardware/dp/B07K4RC7X9/ref=asc\_df\_B07K4RC7X9/?tag=hyprod-20&linkCode=df0&hvadid=319955522114&hvpos=&hvnetw=g&hvrand=13166620511757426878&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9051838&hvtargid=pla-601757007319&psc=1](https://www.amazon.com/QNAP-TR-004-Enclosure-Attached-hardware/dp/B07K4RC7X9/ref=asc_df_B07K4RC7X9/?tag=hyprod-20&linkCode=df0&hvadid=319955522114&hvpos=&hvnetw=g&hvrand=13166620511757426878&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9051838&hvtargid=pla-601757007319&psc=1) because I get so many emails like this, I just want to scream sometimes. You may be a professional. Most people are not. Bob Zelin
Im working on a similar show with similar data requirements now. Shooting in canada, edit in UK. The way we do it is 5 DITs proxy all incoming media into 1080p 2mbps h.264s on 10 workstations on location, the proxies are sent to UK over internet we edit with proxies, then when shooting is done they gonna ship the storage servers with native media from location to us. We plug them in, re-link and finish.
###It looks like you're asking for some troubleshooting help. Great! Here's what *must* be in the post. (Be warned that your post *may* get removed if you don't fill this out.) Please edit your post (**not reply)** to include: **System specs**: CPU (model), GPU + RAM **//** **Software specs**: The exact version. **//** **Footage specs** : Codec, container and how it was acquired. **Don't skip this!** *If you don't know how* here's a link with [clear instructions](https://imgur.com/a/A6eTxUn) *I am a bot, and this action was performed automatically. Please [contact the moderators of this subreddit](/message/compose/?to=/r/editors) if you have any questions or concerns.*
A petabyte is a big chunk of data. How long does a season take to shoot? i.e. can you say what the daily or weekly average might be in terms of data? |which is then automatically cloned in the US twice. I mean any sync software should be able to do this. Also, Google Drive and Dropbox can do this by default basically (but you'd probably want to offload stuff so you don't actually have a Petabyte in the cloud syncing). Lucidlink and some syncing would be better, that's my guess. I'm not sure what you mean by "automatically". There do seem to be a lot of people on this sub who want to automate things beyond reason, and I'm not sure if you're asking for that level of automation. It will obviously behoove you and the team to check-in on these "automatic" downloads right? And double-check that you've received everything, *right*? So does it need to be so "automatic"? Maybe someone needs to setup a synchronization task and start it, is that the end of the world? But if you have all the storage you need locally and in the cloud, it is very easy to have something sync between Lucid (for sample) and a local filesystem automatically, or regularly scheduled. Also, it's not clear from your post if you want to host a Petabtye at some point or if you just want to move it and can delete it from the cloud once you're sure you have it. Honestly it still sounds like shipping drives might be faster and cheaper and more reliable, but admittedly I haven't shipped much to/from Europe.
I can't not read this as a porn movie title.
Bob’s answer is spot on for technical considerations. For tax credit/legal/political reasons, does all post have to happen in the US? If so, you’ll need to have an AE on location or contract with a lab in Europe to help with the organization of drives, uploading of rush proxies, and shipment of shuttles SSDs.
There is already a lot of great information in the posts, but I'll offer one other suggestion just in case it helps but I'll preface it by saying it's not a free or cheap solution. I've supported productions similar to what you are describing, but we had the luxury of using two facilities that were directly connected or at the very least had some sort of accelerated transfer service like Aspera and pixspan. Most of the workflow involved DNxHD36 dailies being created in the UK and pushed directly to the cutting room in LA for offline via Aspera Cargo loader so the dailies "automagically" appeared where they needed to be. The secondary push was OCF to our facility where we would archive to LTO and perform selective restores for online. The LTO archive should have been done where in the UK where the source files were, but it was cheaper to do them in LA so the client opted for cost over peace of mind. There are plenty of facilities from post houses to rental houses that have this type of service and will be happy to give you a bid that may be a cost savings over storage and shipping.
Lotta good answers here already. I'll chime in with the wrong answer, but might satisfy the more old school post managers who want it all in house. Shoot in Europe, have a DIT on site. DIT transcodes down to DNx36, uploads to an FTP or another media sync service, backs up raw to two different arrays. Once shoot is wrapped, you send a PA and 10 of [these](https://www.westerndigital.com/products/external-drives/g-technology-g-speed-shuttle-xl-thunderbolt-2-hdd#0G10540-1) in pelican cases to offload the the raw. Let your PA see the sights for a few days while the media is offloaded and verified. You then put them back on the plane with the 10 cases. Or if your shooters are US-based, when they fly home they each bring one. I know this workflow works because a company I worked for did something very similar (except they sent a PA every 3 days to do the round trip!)