You are here

17 posts / 0 new
Last post
Aperture & NAS Synology DS412+ #1
Niccolò's picture
by Niccolò
April 11, 2014 - 12:45am

Hi Everybody,

I already read other posts (a bit old) about what I’m going to ask. But I wanted to know if any of you could give me an update on the use of a Synology NAS with the “Aperture Library”. 

My situation: 
I have 2 “Libraries Aperture” one of 160GB and the other 70GB of photos, and the files are stored inside the libraries. I read that Apple recommends using the Aperture library only on discs HSF+. 

I would understand if it is possible (without doing damage to the libraries, and to the photos) to move in some way, the photos files (masters) on the Synology NAS partition, and use with good performance and without file corruption, Aperture as I have done to date. 

Thanks to all those who can provide their point of view on this issue. 

Best Regards

NiQ Ph. - Nikon D90 + SB900 + 35mm f1.8 + 50mm f.18 + 105mm f2.8 + 18-135mm f3.4

Chris Biele's picture
by Chris Biele
April 13, 2014 - 2:56pm

Hi Niccolò. I was recently looking into the Synology storage solutions and saw they had some testimonials from Aperture users on their site. After a little more research though I decided to go with a DAS (Direct Attached Storage) instead of NAS because of Aperture’s need for HFS+. From what I could gather, any libraries stored on the Synology would have to be copied to an HFS drive before opening them, and then transferred back. You could theoretically keep referenced originals on the Synology, but there is a high chance of corruption. Is it worth the risk?

I chose a LaCie 4Big RAID, which has had its own set of problems, but I can confidently store and access libraries and referenced originals with peace of mind.

Another thing I noticed is that Scott Kelby has a glowing testimonial on the Synology site, but if you look at his gear list they don’t even get a mention. He uses all G-Tech stuff, so maybe he tried them and then stopped using them for some reason. Obviously he’s a Lightroom user, not Aperture, but it does make you wonder…

Niccolò's picture
by Niccolò
April 14, 2014 - 10:49pm

Hi Chris, thank you for your reply. I watched the LaCie 4Big RAID, but this isn’t what I need, because I need of a really NAS not only for the photos, but also for my music, my movies, my documents. So the photos is only a part of the things that I need to storage.

I performed several tests on a dedicated Aperture library, just to simulate all the steps and I verified the possibility to use “also” a NAS storage with Aperture in the following way.

IMPORTANT! I kept the Aperture library on a HSF+ partition, usually “Images” folder’s user, with a cron/script backup that every night copy in a dedicated position the Aperture library, on the NAS. I used this only for the backup, not for the opening of Aperture.

I start from the situation that I have all the masters inside the Aperture library;

First it’s necessary to “extract” all the master from the Aperture library to a dedicated folder on the NAS storage (Synology, or other) with the function “Relocate Originals for Library” from the “File” menù.

To do this I created a dedicated “export” preset, that I’ll use also for the future imports of photos.(eg. /YEAR/MONTH/DAY/) In my test I chosen to move all the original files in this way, so the Aperture library will contain only the “links” to the master images, and the corrections, and on the NAS I’ll have the masters saved with the following folder structure /YEAR/MONTH/DAY/.

I think that this configuration will work well for me, and maybe will be useful also for other users. Comments are welcomed!

Thank you

Kind regards

NiQ Ph. - Nikon D90 + SB900 + 35mm f1.8 + 50mm f.18 + 105mm f2.8 + 18-135mm f3.4

Chris Biele's picture
by Chris Biele
April 15, 2014 - 1:45am

Hi Niccolò. Interesting setup. So are you currently accessing your originals from the NAS? From what I’ve seen Apple only warns against storing your active LIBRARIES on networked or non-HFS+ drives, so you should be ok… Fingers crossed!

Niccolò's picture
by Niccolò
April 15, 2014 - 1:53pm

Hi Chris, yes this is the configuration that I made. I’ll update the thread if I’ll incurred in some problems! Hope no! :P …

Thank you

NiQ Ph. - Nikon D90 + SB900 + 35mm f1.8 + 50mm f.18 + 105mm f2.8 + 18-135mm f3.4

Guenther's picture
by Guenther
July 15, 2014 - 3:14pm

Hello Niccolò,

Actually I have the same setup, however I have issues with the “Relocate” function. When I execute “relocate” and select the NAS drive (& and Folder) a subfolder is created but then I get the error message “no permissions”. However the user has pemissions. A manual copy works.

 

Do you have any idea? Do you have a HFS Partition on the NAS?

Regards

Günther

Niccolò's picture
by Niccolò
July 15, 2014 - 9:58pm

Hi Günther,

very strange! I can tell you that I’m connected to my NAS with NFS protocol, that allow (or not) to a specific IP (or set) allow the access (or not) to a shared folder with admin (root) privilege.

What is your NAS? What protocol do you use to connect to the NAS?

Bye bye

NiQ Ph. - Nikon D90 + SB900 + 35mm f1.8 + 50mm f.18 + 105mm f2.8 + 18-135mm f3.4

Niccolò's picture
by Niccolò
July 15, 2014 - 10:02pm

PS. Unfortunately I can not create HFS partitions on the NAS (I haven’t found any NAS that natively supports HFS), but only ext4, but using Aperture in this configuration, this shouldn’t affect the library.

Bye

NiQ Ph. - Nikon D90 + SB900 + 35mm f1.8 + 50mm f.18 + 105mm f2.8 + 18-135mm f3.4

Guenther's picture
by Guenther
July 17, 2014 - 10:02am

Hi,

Thanks for the response. In turns out, that the “relocate” works for some projects and for some not. I have not found out if this is because my NAS started indexing in between and that this causes the troubles or if there is anything wrong with the project itself. Need to play around on the weekend :-(

Regards,
Guenther

Jochen H. Schmidt's picture
by Jochen H. Schmidt
July 17, 2014 - 11:08am

I'm using referenced libraries as the active working library. The referenced originals can be on several other drives. I'm using DAS for that, but a NAS is possible too. This allows me to quickly make room on the Mac if it grows too big. I have a snapshot drive (DAS) to which I sync the library file itself over and after that I open the library and consolidate all masters to this library (using “copy” not “move” orginals). So I have a fully “managed” snapshot copy of the library on my snapshot drive.

I archive this snapshot drive and the drive(s) with the referenced originals in a QRecall archive. The cool thing with that is, that QRecall archives have a deduplication on block level. So archiving the managed Aperture library AND the relocated originals doesn't add more space in the archive. I also get a checksum per block so that QRecall can verify and signal any corruption of blocks. As I have multiple QRecall Archives I can then recapture the corrupted files from a valid copy.

Niccolò's picture
by Niccolò
July 18, 2014 - 11:43pm

Hi Jochen,

thank you for your testimony. I did not know the capabilities of QRecall :) …

It seems like a great setup!

 

Guenther,

what is the model of your NAS?

NiQ Ph. - Nikon D90 + SB900 + 35mm f1.8 + 50mm f.18 + 105mm f2.8 + 18-135mm f3.4

Guenther's picture
by Guenther
July 21, 2014 - 10:33am

Hello,

It is a DS414. But I dont think that this is the reason. I could move some projects without a problem. others not. I think this must be something with permissions or so.

 

Will check further.

Regards

Guenther

Rob Phelan's picture
by Rob Phelan
July 22, 2014 - 12:24am

A couple quick points of clarification from a long time creative storage and enterprise data systems guy.

Metadata (It’s not just for photos!):

Aperture requires HFS+ to store a library due to the way it handles file system metadata for items like packageswhich all Aperture libraries are. (just right click on your library and select “Show Package Contents” [ though don’t change anything in there unless you want a borked library]).

In terms of the base file system, your “Library” is just a folder like any other (go look at it on a Linux box). It’s the combination of specialized HFS+ file system metadata, and the Mac’s usage of that metadata, that make it look like a single document that you can double-click and natively open in Aperture.

Connection Protocols & File Systems

So, if the above is true, all we have to do is get a NAS that will format it’s drives as HFS+ right? Nope. It doesn’t matter one iota what the NAS’s drives are formatted with, because all your Mac sees is what is presented by the network transfer protocol you’re using.

For example, most NAS devices support SMB/CIFS (Aka Windows File Sharing) and NFS (Linux File Sharing - broadly stated). Those connection methods provide a layer of abstraction between the client (Your Mac), the host (your NAS) and the base file system (EXT, NTFS, etc). That abstraction layer allows your Mac to actually put data onto an EXT3 file system that it normally wouldn’t have the first clue how to deal with.

The only trouble with that abstraction layer is that it murders (or redistributes) that fancy file system metadata that Aperture loves so much. So when you go to open your Aperture library, it goes ka-blooey.  (AFP [Mac Networking’s file presentation layer] used to be able to handle all that file system metadata for you, but AFP is dead… long live AFP!)

OK, what the heck do I do about it!?

1. Reference your images on your NAS, keep the library on your happy little HFS+ drive.

As others have posted above, your actual image files don’t need any fancy file system metadata. All the metadata they need (EXIF, etc) is actually enclosed in the image files themselves. As such, you can store them on any old file system and reference them via a library living on HFS+.

2. Make an HFS+ island in the middle of your NAS

File systems are funny things, a single file can be use to encapsulate an entire file system. You know those DMG files you download with your Mac installers, that mount on your Mac like a hard drive? Those are actually completely encapsulated HFS+ “Disk Images”. Basically, it’s a little hard drive with all of it’s fancy HFS+ metadata, swaddled into a single container file that doesn’t need any fancy metadata for itself.

That means you can use Disk Utility to create an HFS+ sparse image file (so it can grow on demand), copy your Library into it, dismount the image from your Mac, and copy it to your NAS. Need to open the Library later? Mount the disk image from your NAS (just Double-click), and open your Library from within. All of your library’s fancy metadata is kept safe in the image until it’s mounted on your Mac, at which point it’s all accessible again. This same method can be used to store other metadata-sensitive info (like Time Machine backups) on a NAS or Windows-based server.

It’s not the most efficient thing ever, but it keeps your library and images together, AND all you have to do to backup both is make another copy of your disk image file! Also, backing up a single data file will be MASSIVELY faster (think 10x or better) than trying to copy your library and all it’s associated tiny little previews and files individually across a network.

3. Get your techy on and dive deep!

If your NAS supports iSCSI and multiple partitions of it’s RAID volume, you can always carve out a chunk of the volume and present it directly to your Mac as a logical volume. This means it will be just like plugging in an empty hard drive with no file system. Your Mac will ask you to initialize the “Disk” (LUN), and give you the opportunity to format it as HFS+.

Now, this is effectively turning a corner of your NAS into DAS, as it can only be accessed by one device at a time, it’s not a “Share” like your usual SMB/NFS-based stores. Those are file-level presentation (with their abstraction layer) whereas iSCSI is block-based, and each of those blocks are presented just as they would be on a physical hard drive.

This option isn’t for the faint of heart, and it’s definitely not without it’s drawbacks, but I’m including it here for completeness of discussion.

What about Photos.app? Will this problem still be here?

Apple so far has shown no signs of hedging it’s bets in terms of file systems. So it’s dependence on HFS+ file system metadata is unlikely to end any time soon. Also, given that Photos is supposed to have an upgrade path from Aperture/iPhoto libraries, I suspect that Photos will continue to rely on HFS+’s smorgasbord of file system metadata. Not that it’s a terrible thing, but it does have it’s complications as we’re seeing here. (Sadly I’ve long-since given up that Apple will introduce a modern file system any time soon rather than continuing to extend one that’s been around for decades.)

Any questions?

Hopefully the above info was helpful to the discussion. If anyone finds fault with the explanation, please let me know, I’ll be happy to expound or restate. Meanwhile I’m going to go back to re-writing my Aperture data-mining scripts to be used with any photo app as Aperture’s (very) slow march to the grave continues.

 

Best,

Rob

#nofilter before anyone knew what it meant.

Jonathan Panoff's picture
by Jonathan Panoff
February 27, 2015 - 1:51pm

Hi Rob,

I have a Synology NAS with iSCSI capability. I created a 4TB iSCSI drive to handle all my needs, formatted it in disk utility to HFS+ and then started to build my aperture managed library on it. I then am using an external 4TB hard drive as a vault. The only “problem” I found was that the mac doesn’t have a native iSCSI initiator so had to get a free legacy one from GlabalSAN. Everything seems to be working OK but your input would be appreciated. I do like the idea of having the photos all in one package that is easily backed up instead of having it referenced.One question though. If I wanted to change to a referenced library using the relocate menu option will the directory structure be preserved or will it just be gobbled gook??

Regards

Jonathan

Rob Phelan's picture
by Rob Phelan
February 27, 2015 - 7:31pm

Since you’re dealing with HFS+ on your iSCSI mount, it should work like any other external hard drive, so there’s no reason it should have any trouble referencing photos in another location. 

I always use self-contained libraries, so I don’t have much experience with referencing options, but from a storage perspective, it should cause any problems.

 

-Rob

#nofilter before anyone knew what it meant.

Guenther's picture
by Guenther
July 22, 2014 - 12:52am

Hello,

thanks a lot. This really helps a lot to better understand how this works. However, i could meanwhile solve my issue. I just executed a “repair” on the library and now I can run the “relocate originals”

 

Regards

Guenther

Karthik's picture
by Karthik
December 7, 2014 - 8:20am

HI All:

I am reading about Aperture and NAS and why they can’t work together. Before i stumbled on it, i happened to copy my MAC Data onto the DS215.  Then i happily went over and opened my Photo library from the NAS. Side note - I also have upgraded to a 802.11ac (got the Apple extreme). I see very good performance over the NAS on aperture (but my photo library is only a few 100Gigs). Now reading all this scares me quite a bit… Is the issue that there will be potential data loss or corruption that could happen via this setup? whats the potential mechanism where the corruption can happen?

Thanks 

Karthik

You may login with either your assigned username or your e-mail address.
Passwords are case-sensitive - Forgot your password?