Saturday, March 26, 2011

Working remotely over a high-latency connection

Does anyone have any tips for working remotely with high network latency? My project is Linux/Windows only, so I work remotely on a beastly machine at CSAIL. It works great when I'm at CSAIL, but terribly from Palo Alto. My two solutions thus far have been:

Use vi over ssh. This sucks because vi then occupies my terminal, and all my keystrokes are buffered over ssh so typing is slow.

Use MacVim and OpenAFS (or any other network filesystem here, including sshfs). This works better for me, but when switching files it takes *forever* (> 15 seconds) to tab-complete filenames or open or save a file. I have to believe this is due to inefficiencies in AFS, because I can scp files faster than MacVim can write to AFS.

I think the ideal solution might look something like Dropbox, where I have a full copy of my source code on both machines, and I only wait for the network when I want to go run the tests. However, I'm doing the initial sync now, and it's taking forever, so I'm not sure this is a real solution.

I haven't tried using bcvi or vim's scp:// file protocol. I'm not enthusiastic about them because I tend to use vim like people use emacs, meaning I keep lots of buffers open and open new files from the same vim instance. I want to be able to use tab completion to open new files, and I don't think vim's scp protocol will let me do that, and if it does, it would suffer from the same latency issues.

Ideas?

15 comments:

  1. I've never used it but perhaps http://macfusionapp.org/ is worth looking at.

    ReplyDelete
  2. sshfs might be what you're looking for.

    ReplyDelete
  3. Ironically I believe rsync was written for exactly this purpose to make working on code that needed to be kept in sync between Australia and North America nicer at the time...

    ReplyDelete
  4. Don't: remote filesystems encounter a truly frustrating number of edge cases - even simple things like running ls on a directory cause a number of stat() calls which can't be cached without breaking POSIX and/or losing data. There are a ton of different filesystems, tuneables for implementations like SSHFS, etc. - none of which actually work over more than across-town latency because the real fix would require everything to be rewritten to use non-blocking I/O, which hasn't really gone mainstream outside of web browsers.

    After fighting this for years, I switched to a DVCS model where I use a local Git repo and push changes out to the remote system for testing - the main advantage being that it only sends deltas and is even reasonably efficient for large binary files which get append-only changes. I believe Dropbox uses similar delta-handling algorithms so your initial painful sync should be much worse than normal but I haven't seen how well it would handle simultaneous changes on both sides.

    ReplyDelete
  5. Two options:

    1. Use NX (http://nomachine.org/). This only works on Linux (X11), but it's how I did most of my PhD work and it lets you run a real graphical editor window (or 5) with latency acceptable enough to work over a modem or cellular data connection if necessary. If you find SSH too high-latency, this may still be too obnoxious for you though.

    2. Just bite the bullet, run a Linux/Windows VM locally to prototype, and use a DVCS or rsync to run things for real. That's what I'm doing right now; I have a Windows VM running in the background and have its drive mounted on my Mac so I can use editors, the command line, etc. on the Mac side wherever possible.

    ReplyDelete
  6. You could still use bcvi to support your preferred way of working. By default it runs 'gvim scp:...' but you could add vim's --remote option to have it open the file in an existing editor process.

    Of course the bcvi workflow assumes that you have a shell open on the remote server. Using the SSH 'ControlMaster' config option to reuse an existing SSH connection (see link) will significantly reduce latency. If you don't have an existing shell connection open then there's not likely to be any solution to your latency problem.

    http://sshmenu.sourceforge.net/articles/transparent-mulithop.html#reusing-connections

    ReplyDelete
  7. Thanks for sharing your info. I really appreciate your efforts and I will be waiting for your further write ups thanks once again.
    Vee Eee Technologies| Vee Eee Technologies|

    ReplyDelete
  8. very nice thanks for sharing

    hey friend see snow on google
    Type “Let It Snow” on @Google If you click and drag you can wipe the snow away. It is great. source: http://le-titsnow.blogspot.com

    ReplyDelete
  9. Hi Reid Kleckner do you have any official site... if you have share with us please...
    Placement Consultants in Chennai

    ReplyDelete
  10. Considerably, the post is actually the best on this laudable topic. I fit in with your conclusions and will eagerly look forward to your upcoming updates. Just saying thanks will not just be enough, for the tremendous lucidity in your writing. I will right away grab your rss feed to stay informed of any updates. Good work and much success in your business enterprize!





    No Addiction | Aire Bra | Sandhi Sudha | Fair Look |
    sandhi sudha oil | Body Buildo Powder | Step Up Height Increaser | Hair Building Fiber | Slim 24 Pro

    ReplyDelete
  11. As this website majorly give importance to the private jobs as well as govt jobs related information we appreciate it for the quality of its content.

    ReplyDelete

  12. Recruitment Notification is one of the Top most Job Portal Tamilnadu govt jobs 2017there

    ReplyDelete
  13. Thank u for Sharing this Information. For more cashback Offers And cashback coupons just ping me

    ReplyDelete