Live Mesh Review: Big Mistake! Back to Allway Sync

,

I hate keeping things simple or easy, so in order to always retain a certain level of complication and frustration in my life, I insist on using two computers, a desktop and a laptop. A smart person would just buy a really powerful laptop and be done with it. Too easy. The one plus of this system, though, is that by keeping them in synch, I basically have two versions of every file that matters, plus my most recent archive on an external hard drive.

Since this is a long review of my trials and travails with Live Mesh, let me give a little summary at the top for the impatient types and the pros and cons of Live Mesh.

Live Mesh pros

  • Synchronization through your internet connection, so it does not depend on the reliability of your home network.
  • Real-time synchronization that works in the background so you don’t have to remember to sync.
  • Off-site storage (5GB) so you can actually sync while one computer is off.

Live Mesh negatives

  • Data integrity and potential data loss. Live Mesh doesn’t use a transactional model, or anything close, and deletes data willy nilly while it sorts out your supposed conflicts (see the "Cardinal Sin" section at the end).
  • Performance. Synchronization through an internet connection is very slow. It will take days to synchronize an amount of data that you will synch in minutes through the local connection.
  • System performance. My system ground to a halt while Live Mesh ran, eventually getting to the point where it took over four minutes to launch Word. It normally takes under two seconds.
  • System stability. After a couple of days, both computers running Mesh started to experience major issues. Application crashes, system instability and lockups, system crashes. These computers were formerly quite stable and restabilized as soon as I removed Live Mesh.
  • Phantom conflicts. I created perfect mirrors in the file systems for the folders I was planning to synch. Same files, same timestamps. Mesh flagged almost every one of them as a conflict. I’m not sure how Mesh determines whether or not there’s a conflict, but there is a serious problem with the algorithm. The two other synch/compare tools I ran found no conflicts.

Bottom Line

A friend is running Live Mesh with great success so I know it can work for some people, but for me it was a disaster. I have previously depended on two excellent, stable and reliable tools, Allway Sync and Beyond Compare, and I’m back to using those and that’s what I still recommend. Live Mesh has, in my opinion, fundamental architecture flaws that simply can’t be solved by any number of bug fixes. I’d like to see a product like Live Mesh, but properly implemented.

My Live Mesh Saga in Detail

So, this long saga of my Live Mesh trial (in at least two sense of that word) may try the average reader. If you want the real meat of it, skip to the Cardinal Sin section at the end.

The Problem

The hard part is keeping the computers in synch. For normal daily syncs, I have been using Allway Sync, which is a great product. For more complicated syncs and anything involving synching a local machine with data on a server, I use Beyond Compare, which is a file/directory comparison tool, but has some great synchronization tools (I need to review these, because they’re worth knowing about). The Achilles heel of Allway Sync and Beyond Compare, and almost every similar tool, is that they depend on my local network. Unfortunately networking my Windows XP Home computer and my Vista Home Premium computer is an exercise in frustration, with the Vista computer unable to see the XP computer ever, and the XP computer losing the connection to the Vista computer frequently. Also, these syncs only happen when I remember to do them, rather than happening in the background. That means I don’t waste system resources on background processes, but I could get caught with my data down, so to speak.

Enter Live Mesh

In theory, Live Mesh would solve both of the problems. It does it’s synching through the regular internet connection and it synchs files whenever a file is created or modified. For the first 5GB of data, it also stores the data on your Live Mesh Desktop, which is on a remote Microsoft server. 5GB is kind of laughable by today’s standards (I commonly file a 4GB photo card on a weekend hike), but it’s more than zero. Then finally, Live Mesh includes a remote desktop tool that gives you access, upon approval, from one computer to the other, and unlike the solutions that run over the local network, it worked from the Vista machine to the XP machine as well as the other way. Sounds like a great solution. And it’s a Microsoft product. I know for many people that means run for the hills, but I figured if anyone should be able to get something to work stably with the two latest Miscrosoft operating systems, it should be Microsoft. Figure again.

Getting Started — Initial Quibbles

I found getting Live Mesh up and running rather unintitive. Before installing it, I had done an up to the minute sync and so I knew both file sets were identical. It took some playing around to figure out how to tell Mesh which folders to match with which folders. On two machines with identical file structures, that might not be a problem, but since XP has My This and My That (e.g. My Documents) and Vista has just This and That (e.g. Documents) the file paths do not line up. So what Mesh likes to do is create a new folder on the target machine desktop that correlates to the source folder on the machine you’re working on.

The trick is that when you add a folder to Mesh on one machine, tell it to synch with the other machine (or was it not synch?). Then go to the target machine, right click the light blue folder on the desktop and tell Mesh which local folder to match that to. If the folder is dark blue, it’s too late, you can’t change it and you have to delete the Mesh folders and start over again. And again. I repeated this several times before I got it right. If that sounds confusing and unnecessarily complex for what should obviously be the simplest use case and primary task for Mesh, you’re starting to get why this is not a positive review. Eventually, though, I got all that figured out after much trial and error and Mesh started its work.

And then I started looking for missing features I had come to expect from Allway Sync

  • File masks — omit certain types of files from my synch because I don’t want to copy, for example, shortcuts from one machine to another because XP and Vista have different file paths. Live Mesh is all or nothing.
  • Folder masks — omit some subfolders from synch. I keep my archived backups, for example, only on the desktop (and external hard drive). Again, Live Mesh is all or nothing.
  • Force a synch — this isn’t part of the Mesh paradigm. It’s real-time synching. The thing is, 90% of the time when I use the laptop, I don’t have an internet connection. So when I’m getting ready to head up to the mountain house for a few days, I like to force a quick synch before I head out the door. In theory, Mesh is up to date at all times, but that’s only true if you have a connection between the machines at all times. That might be good for an office worker who is synching with colleagues or a file server, but it’s not my situation.

Signs of Trouble

Now, if these two machines are in synch, it takes Allway or Beyond Compare about 5 minutes to run the analysis and generate a report. Unless I’ve just uploaded 10GB of photos or unzipped some massive open source application with thousands of files, the actual synch takes a couple of minutes. Usually I allow ten minutes total, but half of that if I know they’re basically already mirroring each other.

So Mesh started working. I expected that the internet synch over 1.5Mbps DSL would be slower than over the local network, but I was completely unprepared for this. Even though the folders in question were identical, Mesh was working for days and there is no progress indicator, so I had no idea whatsoever whether or not I was even getting close. Granted, I was synching about 50GB of data, but every file was bit for bit identical with identical timestamps. How hard could this be?

Then It Gets Bad: Crashes, Slowdowns and Conflict Hell

Apparently, really hard. As Live Mesh ground away, my Vista computer became progressively slower and less stable. Applications crashed. Live Mesh crashed and had to be restarted several times. It got to the point where I couldn’t work. I counted four minutes and two seconds to launch Word. And it wasn’t Word’s fault. Every app started behaving like that. And Word started to get a little crazy and at one point I could not save documents. Even closing and reopening Word (four minutes remember), I could not save a document. Finally, I shut Mesh off temporarily and Word sprang back to life, launching in under two seconds, but I actually had to reboot to get file-saving abilities back. So I decided I could only let Mesh run while I wasn’t actually doing anything. Eventually, though, Mesh crashed and would not restart on the Vista machine.

Meanwhile, it was cranking along the laptop and I decided to let it run and see what woud happen. What happened is I started getting notifications that I was out of disk space and that computer was crashing too. Now, I only had 22GB of free space on that machine, but since I was not adding one single byte of data, I figured that would be okay. Not quite. In two days, Mesh had brought down two computers, both of which had run reliably for years.

So clearly, I had to get Mesh off my system and uninstall it. So I did. That was easy enough. Now I had to figure out how to get things back the way they were. The first thing I noticed is I had a Live Mesh folder on my desktop. I looked inside and it had a subfolder labelled “conflicts”. Hmm… that’s interesting. Why would there be any conflicts? I wonder what’s in there. Answer: thousands of files. A total of something like 18GB of zip-compressed data. Now, that’s enough for me to damn Live Mesh to the Seventh Circle of Software Hell. Why is it finding conflicts when the data is bit-for-bit identical, has the same hash, and has the same timestamp? I mean, for God’s sake, what other tests are there? What possible conflict is there?

Essentially, Live Mesh saw every single file it analyzed as a conflict, and that was what was filling up my disk — when it sees a conflict, it puts a zipped copy of the file in a Live Mesh folder to await resolution (or so I thought). So I deleted that folder to get the XP machine running again.

The Cardinal Sin: Not Respecting Data Integrity

Then I fired up Allway Sync to analyze the disks in case Mesh had changed any data. Oh, my naiveté! Thousands and thousands of deleted and new files. So then I fired up trusty Beyond Compare for a little more analysis. It turns out that every time Live Mesh had found a conflict it had deleted one copy and moved, not copied, that file to the Conflicts folder. In the meantime, it had created thousands of placeholder files with the name filename.ext.xlp.

I don’t think I lost any data, because Mesh keeps one copy and deletes the other, but I had to go through and delete thousands of *.xlp files and then essentially move 27GB of data (the uncompressed size of the 18GB of compressed conflicts) from the computer that still had a copy to the one that didn’t (not always the same).

So here’s the cardinal sin: any data synch like this should be as transactional as possible. In SQL, you can group operations into transactions, which means that a set of operations fails if any individual operation fails. Here’s an example. When you go to the ATM machine and ask for money, the transaction looks like this:

  • Request money from account
  • Debit account for amount
  • Spit bills out the slot
  • Roll up the transaction

If the machine fails to give you the bills, the transaction fails, the debit gets rolled back and you don’t have to prove you didn’t get your money. The financial system and any system that depends on having the data right depends on a transactional model.

The key principle is data integrity. You can get a lot wrong in an application, but you shouldn’t screw up the data integrity part. So in my opinion, the fatal and fundamental flaw of the Live Mesh architecture is that it does a poor job of maintaining data integrity. Now, data synch can’t be fully transactional, because if the connection dies, you can end up unable to roll the transaction back. But, the placeholder files that mark the fact that Mesh has flagged a conflict should be in the Live Mesh/conflicts folder and the original data should be left intact for as long as possible. In other words, it should identify the conflict, which presumably requires user intervention of some sort, and do nothing to the data until it has instructions from the user. The Desktop/Live Mesh/conflicts folder should hold the placeholders and leave the original files alone until the last possible moment. To go around deleting my data and creating placeholder files in my data folders merely to mark the existence of a conflict is borderlin criminal!

So for me, the broad outlines and concept for Live Mesh is interesting, but the implementation is so fundamentally flawed that I don’t think any amount of bug fixes could redeem it and convince me to use it and I can’t recommend it.

8 Responses to “Live Mesh Review: Big Mistake! Back to Allway Sync”

  1. Thank’s Tom for this report. As you know, I’m a multiple computer user (not at the same time ;-) ) and always looking to have my data up to date on each one…
    So I’ll not spend time on trying Mesh…

    With my best wishes, Dan.

  2. Hello!

    Thanks for the interest in our synchronization software. We are glad that it is convenient for you and very pleased with this review.

    Best regards,
    Allway Sync Team.

  3. After reading your blog I checked out Allway Sync.

    I’m definitely not it after reading this from their site below (see faq question below).

    Rather than have their programmer check for memory usage and avoid a crash they want you to organize your files a certain way to not crash.

    I could live with a bug no problem, but the fact they thought this was acceptable enough to make it an FAQ is scary.

    Allway FAQ – Question:
    “doing a massive backup of XXX,000 files … crashes. Why?”

    Allway FAQ – Answer:
    “Our sophisticated sync algorithm … can result in the consumption of a lot of memory … Eventually, memory runs out and the program can’t continue. … try to keep files that might be modified (and so need to be synchronized) separate from those that are static (software distributions, program folders) and exclude the latter from synchronization.

  4. My goal isn’t to sell anyone on Allway. It works for me that’s all I can say. I mostly want to warn people about Live Mesh.

    That said, here’s what I can say from *my* experience with Allway.

    Unlike Live Mesh, when Allway runs into a problem, there’s no data loss. The worst that happens is that if you move a file, it counts that as a delete and then a new file (of course), so if the delete comes first, and then the synch is interrupted, you’ll only have one copy. If you restart the synch, you’ll have both copies. That’s unavoidable and acceptable. The way Live Mesh does it is not.

    And I regularly run synchs with tens of thousands of files (48,000 in my standard end of the day synch set) without a problem and my computer doesn’t really even slow down, and it doesn’t seem to have excessive memory usage (the machine has 2GB of memory). My guess is that you would need to be synching huge numbers of files on a machine without a lot of free memory.

    Finally, they are not asking you to “organize your files in a certain way”. You can handle this either via organization (i.e. your programs are in the Program Files folder) or you can handle it by using filters so that folders that contain OS files or apps aren’t part of the synch (which you usually don’t want anyway). In any case, it just takes a little longer to set up the synch.

    If you know an amazing Windows-based synch app, please add it to the comments.

  5. sugarsync … …other than the shakey at times support, it is the best around. I sync over 200 GB of files between three windows computers, and just my images on a mac. Can see, email, edit and share via my HTC touch pro (running wm6) … though i think there is a version for iphone as well.

    enough said.

  6. Hmm… Sugarsync would cost me $150/year for the amount of data I sync. $250 if I wanted to go big like you. It just isn’t worth $250/year for me at this point.

    I’m thinking about it though. Having backups in the cloud is the future. I see keeping your own backups currently as the equivalent of keeping your money in the mattress. Though trust in banks has taken a hit, most people feel much safer with their money in the bank and their data at home. That’s a measure of the infancy of data backup services.

    I suspect that in 20 years, keeping terabytes of data in your home awaiting thieves and fires will seem as foolish as keeping your life savings under your bed.

    Either that or bank failures will result in everyone investing in massive home vaults that will hold data and money… Long term, though, they’ll both end up in a similar place!

  7. Thanks for a detailed review on the service, Tom. I was about to signup for it but good sense (and past experience with Windows reliability) made me Google for some user experiences. And, boy, am I glad I came across this writeup!

Leave a Reply

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>