Integrating with Spotlight

The issue, as mentioned, is one of multiple files. Ideally, for our recipe application, we want one Spotlight “record” for each recipe in our Core Data repository. For Spotlight to work properly, we’d need one file on the disk for each recipe, along with its associated metadata. Therefore, to make Spotlight happy, we’ll do exactly that. However, since all our data is being stored in a Core Data repository, there’s no reason to store any data in these files. These additional files exist purely for Spotlight (and Quick Look) to utilize. Since Spotlight doesn’t need any data in the files to work (it just needs metadata), we’ll create very simple files and link them back to our Core Data repository.

The other gotcha with Spotlight is that the importer needs to be as fast as possible. What might be acceptable for processing one file or ten files isn’t going to fly when Spotlight has to chug through thousands of files. Since the same importer that we’re writing for use inside our application could potentially be used in a server situation, it needs to be as fast as we can make it. So, we’re going to cheat a bit. Instead of looking up the metadata in our Core Data repository upon request from Spotlight, we’ll store the metadata in the files we’re creating for Spotlight. That way, our importer has to touch the metadata files only and doesn’t need to initialize the entire Core Data “stack” (that is, NSManagedObjectContext, NSPersistentStoreCoordinator, and NSManagedObjectModel).

Creating the Metadata Files

We first need to produce and update the metadata files on the fly. To keep them as simple as possible, we just use plist files, as opposed to a binary representation or some other format. Since NSDictionary understands plist files, it reduces the amount of overhead needed for loading and saving the files.

To begin, let’s create our first NSManagedObject subclass. This subclass handles producing the NSDictionary that will contain all the metadata. Since we’re creating a subclass, we might as well implement some of the properties we’ll be using to reduce the code complexity and make it easier to maintain.

Therefore, our header file looks as follows:

Spotlight/PPRecipe.h
 #import <Cocoa/Cocoa.h>
 
 #define kPPImagePath @"kPPImagePath"
 #define kPPObjectID @"kPPObjectID"
 #define kPPServes @"kPPServes"
 #define kPPItemTitle (id)kMDItemTitle
 #define kPPItemTextContent (id)kMDItemTextContent
 #define kPPItemAuthors (id)kMDItemAuthors
 #define kPPItemLastUsedDate (id)kMDItemLastUsedDate
 #define kPPItemDisplayName (id)kMDItemDisplayName
 
 @interface​ PPRecipe : NSManagedObject
 
 @property​ (assign) NSString *desc;
 @property​ (assign) NSString *name;
 @property​ (assign) NSString *type;
 @property​ (assign) NSManagedObject *author;
 @property​ (assign) NSDate *lastUsed;
 
 -​ (NSDictionary*)metadata;
 -​ (NSString*)metadataFilename;
 
 @end

We need to make sure we change the Class setting in the latest data model so Core Data uses our subclass rather than the default NSManagedObject as shown in the screenshot.

images/ClassChangeForRecipeEntity.png

Implementing the Metadata Method

The goal of this metadata file is to contain just enough information to populate Spotlight and Quick Look but not so much information that the files become large and cumbersome. We must pretend there will be thousands of these files (even if in reality that would be impractical), and we don’t want to impact the users’ performance or their hard drive capacity. For our metadata files, we really need only the following information:

Most of that list is very light—just text. However, the image is probably too large to cram into the plist file, especially because we can’t be sure how large that file will be. In addition, it’d complicate the file format by including binary data. Therefore, we’ll put in the path of the image instead of the actual image. Since the image is stored on disk, we just access that copy.

In addition to this list, we need to add one more item that is not user facing. We want a way to link back to the recipe record in our Core Data repository so if the user tries to open the metadata file, instead our application will open and select the correct record. To do this, we use the NSManagedObjectID of the recipe and store its URIRepresentation (which is actually an NSURL) as a string in the metadata.

Spotlight/PPRecipe.m
 -​ (NSDictionary*)metadata;
 {
  NSMutableDictionary *metadataDict = [NSMutableDictionary dictionary];
  metadataDict[kPPItemTitle] = [self name];
  metadataDict[kPPItemTextContent] = [self desc];
  metadataDict[kPPItemAuthors] = [[self author] valueForKey:​@"name"​];
  metadataDict[kPPImagePath] = [self valueForKey:​@"imagePath"​];
  metadataDict[kPPItemLastUsedDate] = [self lastUsed];
  metadataDict[kPPServes] = [self valueForKey:​@"serves"​];
  NSString *temp = [NSString stringWithFormat:​@"Recipe: %@"​, [self name]];
  metadataDict[kPPObjectID] = temp;
  temp = [[[self objectID] URIRepresentation] absoluteString];
  metadataDict[(id)kMDItemTitle] = temp;
 return​ metadataDict;
 }

Implementing the metadataName Method

Because we want users to be able to view the actual metadata files in the Finder, the filenames should represent the recipe rather than an abstract name. We use the name attribute of the recipe itself as the filename.

Spotlight/PPRecipe.m
 -​ (NSString*)metadataFilename;
 {
 return​ [[self name] stringByAppendingPathExtension:​@"grokkingrecipe"​];
 }

Generating and Updating the Metadata Files

Now that we have an implementation for generating the metadata per recipe, we need to add the ability to populate these files and keep them up-to-date. Ideally, we want to refresh the metadata files every time that the NSManagedObjectContext is saved. To do this, we add a new ‑save: method to our AppDelegate and route all of our saves through it.

Spotlight/AppDelegate.m
 -​ (BOOL)save:(NSError**)error;
 {
  NSManagedObjectContext *moc = [self managedObjectContext];
 if​ (!moc) ​return​ YES;
 
 if​ (![moc hasChanges]) ​return​ YES;
 
 //Grab a reference to all of the objects we will need to work with
  NSSet *deleted = [moc deletedObjects];
  NSMutableSet *deletedPaths = [NSMutableSet set];
 for​ (NSManagedObject *object ​in​ deleted) {
 if​ (![object isKindOfClass:[PPRecipe class]]) ​continue​;
  [deletedPaths addObject:[object valueForKey:​@"metadataFilename"​]];
  }
 
  NSMutableSet *updated = [NSMutableSet setWithSet:[moc insertedObjects]];
  [updated unionSet:[moc updatedObjects]];
 
 //Save the context
 if​ (![moc save:error]) {
 return​ NO;
  }
 return​ [self updateMetadataForObjects:updated
  andDeletedObjects:deletedPaths
  error:error];
 }

In this new ‑save: method, we’re doing a couple of things before calling ‑save: on the NSManagedObjectContext. Because the NSManagedObjectContext knows what objects have been deleted, updated, or inserted, we want to grab a reference to that information before the ‑save: occurs. Once the ‑save: is complete, that information is no longer available. Therefore, we grab a reference to the NSSet of deleted objects, updated objects, and inserted objects. Because the deleted objects will be, well, deleted once the ‑save: is performed, we want to extract the information we care about beforehand. So, we loop over the deleted objects looking for Recipe instances. When we find one, we extract its metadataFilename and store it in a new NSMutableSet. In addition, since we’ll be doing the same thing to the inserted and the updated objects, we merge them into one set. Once we have that information, we go ahead and save the context. If the save fails, we just abort and let the calling code handle the error. When the save is successful, it’s time to update the metadata.

Spotlight/AppDelegate.m
 if​ ((!updatedObjects || ![updatedObjects count]) &&
  (!deletedObjects || ![deletedObjects count])) ​return​ YES;
 
 NSString *path = [self metadataFolder:error];
 if​ (!path) ​return​ NO;
 
 BOOL directory = NO;
 
 NSFileManager *fileManager = [NSFileManager defaultManager];
 if​ (![fileManager fileExistsAtPath:path isDirectory:&directory]) {
 if​ (![fileManager createDirectoryAtPath:path
  withIntermediateDirectories:YES
  attributes:nil
  error:error]) {
 return​ NO;
  }
  directory = YES;
 }
 if​ (!directory) {
  NSMutableDictionary *errorDict = [NSMutableDictionary dictionary];
  NSString *msg = NSLocalizedString(​@"File in place of metadata directory"​,
 @"metadata directory is a file error description"​);
  [errorDict setValue:msg forKey:NSLocalizedDescriptionKey];
  *error = [NSError errorWithDomain:​@"pragprog"
  code:1001
  userInfo:errorDict];
 return​ NO;
 }

Because we want to be in the habit of assuming nothing, we first check that there’s something to update or delete. Once we’re past that check, we need to confirm that the cache directory is in place, and either our metadata directory is in place or we can create it. If any of this fails, we update the NSError object and return.

Spotlight/AppDelegate.m
 NSString *filePath = nil;
 if​ (deletedObjects && [deletedObjects count]) {
 for​ (NSString *filename ​in​ deletedObjects) {
  filePath = [path stringByAppendingPathComponent:filename];
 if​ (![fileManager fileExistsAtPath:filePath]) ​continue​;
 if​ (![fileManager removeItemAtPath:filePath error:error]) ​return​ NO;
  }
 }

The next part of updating the metadata is to remove any files that are no longer appropriate. Therefore, if the passed-in deletedObjects set contains any objects, we need to loop over it. Since we know that the name of the metadata file is stored in the deletedObjects variable, we append it to the metadata directory path and check for the existence of the file. If it exists, we delete it. (It may be possible that a recipe got created and deleted without ever being saved to disk. It’s unlikely, but why take chances?) If we run into an issue deleting the file, we abort the update and let the calling method handle the error.

Spotlight/AppDelegate.m
 if​ (!updatedObjects || ![updatedObjects count]) ​return​ YES;
 
 NSNumber *_YES = [NSNumber numberWithBool:YES];
 NSDictionary *attributesDictionary = [NSDictionary
  dictionaryWithObject:_YES
  forKey:NSFileExtensionHidden];
 for​ (id object ​in​ updatedObjects) {
 if​ (![object isKindOfClass:[PPRecipe class]]) ​continue​;
  PPRecipe *recipe = object;
  NSDictionary *metadata = [recipe metadata];
  filePath = [recipe metadataFilename];
  filePath = [path stringByAppendingPathComponent:filePath];
  [metadata writeToFile:filePath atomically:YES];
  NSError *error = nil;
 if​ (![fileManager setAttributes:attributesDictionary
  ofItemAtPath:filePath
  error:&error]) {
  NSLog(​@"Failed to set attributes: %@​​\n​​%@"​,
  [error localizedDescription], [error userInfo]);
  abort();
  }
 }
 
 return​ YES;

The last part of updating the metadata files is to process existing or new recipes. As with the deleted objects earlier, we first check to see whether there are any objects to update, and if there aren’t, we’re done. If there are new or updated objects, we again loop through the NSSet looking for PPRecipe entities. For each recipe we find, we request its metadata NSDictionary object from the metadata method we created earlier. Using that NSDictionary along with the metadataFilename method, we write the NSDictionary to disk. For one last bit of polish, we update the attributes on the newly created (or updated) file and tell it to hide its file extension. This gives us the cleanest appearance when viewed inside the Finder.

Now that the ‑save: method has been written, we need to route all the ‑save: calls that exist to call this method instead of calling ‑save: directly on the NSManagedObjectContext. Currently, this requires modifying both the ‑(NSApplicationTerminateReply)applicationShouldTerminate: method and the ‑(IBAction)saveAction: method. In each case, we just need to change the following:

 [[self managedObjectContext] save:&error];

to a message to the ‑save: method on the AppDelegate.

 [self save:&error];

There’s one last situation we have to handle. If we have existing users and are adding the Spotlight integration after v1.0, we need some way to bring our users up to speed. To do this, we add a check to the ‑(void)applicationDidFinishLaunching: method. If the metadata directory doesn’t exist, we must do a full push of all the metadata in the persistent store.

Spotlight/AppDelegate.m
 NSError *error = nil;
 NSString *path = [self metadataFolder:&error];
 if​ (!path) {
  NSLog(​@"%s Error resolving cache path: %@"​, __PRETTY_FUNCTION__, error);
 return​;
 }
 if​ ([[NSFileManager defaultManager] fileExistsAtPath:path]) ​return​;
 
 NSManagedObjectContext *moc = [self managedObjectContext];
 NSFetchRequest *request = nil;
 request = [NSFetchRequest fetchRequestWithEntityName:​@"Recipe"​];
 
 NSSet *recipes = [NSSet setWithArray:[moc executeFetchRequest:request
  error:&error]];
 if​ (error) {
  NSLog(​@"%s Error: %@"​, __PRETTY_FUNCTION__, error);
 return​;
 }
 [self updateMetadataForObjects:recipes andDeletedObjects:nil error:&error];
 if​ (error) {
  NSLog(​@"%s Error: %@"​, __PRETTY_FUNCTION__, error);
 return​;
 }

Here we’re looking for the metadata cache directory, and if it doesn’t exist, we fetch every recipe entity in the persistent store and pass the NSSet to our metadata-building method. This also protects us from users who like to periodically delete their cache directory. This method calls the ‑metadataFolder method to determine where the metadata should be stored.

Spotlight/AppDelegate.m
 -​ (NSString*)metadataFolder:(NSError**)error
 {
  NSString *path = nil;
  path = [NSSearchPathForDirectoriesInDomains(NSCachesDirectory,
  NSUserDomainMask, YES) lastObject];
 if​ (!path) {
  NSMutableDictionary *errorDict = [NSMutableDictionary dictionary];
  NSString *desc = NSLocalizedString(​@"Failed to locate caches directory"​,
 @"caches directory error description"​)
  [errorDict setValue:desc forKey:NSLocalizedDescriptionKey];
  *error = [NSError errorWithDomain:​@"pragprog"
  code:1000
  userInfo:errorDict];
 return​ nil;
  }
  path = [path stringByAppendingPathComponent:​@"Metadata"​];
  path = [path stringByAppendingPathComponent:​@"GrokkingRecipes"​];
 return​ path;
 }

In the ‑metadataFolder, we first request a list of the cache directories from the NSSearchPathForDirectoriesInDomain method and append the path components Metadata and GrokkingRecipes to it. We don’t check to see whether the path exists at this point but instead let our caller decide how to handle that.

Creating the Spotlight Importer

Now that we have some metadata to work with, it’s time to build the Spotlight importer. To start this part of the application, we need to first address UTIs.

Uniform Type Identifiers (UTIs)

Both Spotlight and Quick Look use UTIs rather than filename extensions to connect files on disk with (Spotlight) importers and (Quick Look) generators. A UTI is a unique string that identifies the type of data stored in a given file. It’s recommended that UTIs identify the company and application that created the data file, and like bundle identifiers, a reverse domain name is ideal for this purpose. (It should be noted that bundle identifiers are in fact UTIs themselves.) Since our application uses com.pragprog.grokkingrecipes as its unique bundle identifier, we’ll use the same UTI as the value of the LSItemContentTypes to identify the files.

Spotlight/Info.plist
 <key>CFBundleDocumentTypes</key>
 <array>
  <dict>
  <key>CFBundleTypeExtensions</key>
  <array>
  <string>grokkingrecipe</string>
  </array>
  <key>CFBundleTypeIconFile</key>
  <string>book.icns</string>
  <key>CFBundleTypeName</key>
  <string>Grokking Recipe</string>
  <key>CFBundleTypeRole</key>
  <string>Editor</string>
  <key>LSItemContentTypes</key>
  <array>
  <string>com.pragprog.grokkingrecipe</string>
  </array>
  <key>NSPersistentStoreTypeKey</key>
  <string>XML</string>
  </dict>
 </array>

The UTExportedTypeDeclarations section is probably very familiar. Xcode generates it to describe any file that’s handled by the application being built. The one difference is that, instead of defining a file extension (like txt), we’re defining a UTI that is being handled by our application. Since this UTI is unknown by the system, we need to describe it, again in our Info.plist file.

Spotlight/Info.plist
 <key>UTExportedTypeDeclarations</key>
 <array>
  <dict>
  <key>UTTypeConformsTo</key>
  <array>
  <string>public.data</string>
  <string>public.content</string>
  </array>
  <key>UTTypeDescription</key>
  <string>Grokking Recipe</string>
  <key>UTTypeIdentifier</key>
  <string>com.pragprog.grokkingrecipe</string>
  <key>UTTypeTagSpecification</key>
  <dict>
  <key>public.filename-extension</key>
  <string>grokkingrecipe</string>
  </dict>
  </dict>
 </array>

This key describes exporting our UTI and tells Mac OS X how to link it to different file extensions. In addition, this section describes the data to Mac OS X, telling the OS a descriptive name for the data type and where in the UTI tree it fits.[5]

Xcode Subproject

Our Spotlight importer is actually its own application. Xcode handles this with a separate project for the importer. (It’s actually possible to include the plug-in as part of the main application project, but I’ve found that to be more hassle than it’s worth.) Since we want to include the importer as part of our primary application and we don’t want to have to remember to rebuild the subproject every time we build our main project, we’ll set it up as a dependent or subproject within our primary project. To do this, we start with creating a project in Xcode and selecting the Spotlight importer, as shown in the following image.

images/SpotlightNewProject.png

We want to save this project in a directory inside our primary recipe project, and we don’t want to be too clever. We’ll give the subproject an obvious name like SpotlightPlugin and include it with the Spotlight example project. To make Xcode build this plug-in every time we build the main project, we need to link the two together.

images/CreatingAProjectDependency.png

This is accomplished with the following steps:

  1. Drag the subproject into the main project (shown in the previous figure).
  2. Open the target in the main project, and select the General tab.
  3. Add the subproject as a dependency.
  4. Add a new copy phase to the main project’s target, and set its destination to wrapper and path to Contents/Library/Spotlight.
  5. Drag the Spotlight plug-in into the new build phase (shown next).
images/CopyPluginToBuildPhase.png

Now, whenever we clean or build the main project, the subproject is cleaned/built. Taking this step also allows the subproject to be built with the same settings as the primary project.

Linking the Spotlight Importer to the UTI

With our Spotlight importer subproject in place, it’s time to link the importer to the UTI for our metadata files. To do this, we need to update the Info.plist of our Spotlight subproject to let the operating system know which UTIs this importer handles.

 <array>
  <dict>
  <key>CFBundleTypeRole</key>
  <string>MDImporter</string>
  <key>LSItemContentTypes</key>
  <array>
  <string>com.pragprog.grokkingrecipe</string>
  </array>
  </dict>
 </array>

Here, we’re defining our plug-in as having an MDImporter role, and the list of UTIs contains just the one for our metadata file. With this change, Mac OS X knows to use this importer to retrieve the information for our metadata files.

Building the Spotlight Importer

Now that everything is connected, it’s time to build the importer. Fortunately, this is the easiest and shortest part of the entire process. The Spotlight template created the main.m file that we’ll be using, and it contains all the boilerplate code for us. The only code we need to write for the importer is in the GetMetadataForFile.m file. The template generates a GetMetadataForFile.c file, and that file won’t accept any Objective-C code. Since I prefer Objective-C over straight C, the first thing I did was rename the c file to an m file. This tells Xcode to compile it as Objective-C rather than C. Since we’ll be using Foundation APIs, we need to include Foundation.framework as well.

Spotlight/SpotlightPlugin/GetMetadataForFile.m
 #include <CoreFoundation/CoreFoundation.h>
 #include <CoreServices/CoreServices.h>
 
 #import <Foundation/Foundation.h>
 
 Boolean GetMetadataForFile(​void​* thisInterface,
  CFMutableDictionaryRef attributes,
  CFStringRef contentTypeUTI,
  CFStringRef pathToFile)
 {
  NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
  NSDictionary *meta;
  meta = [NSDictionary dictionaryWithContentsOfFile:(NSString*)pathToFile];
 for​ (NSString *key ​in​ [meta allKeys]) {
  [(id)attributes setObject:[meta objectForKey:key] forKey:key];
  }
  [pool release], pool = nil;
 return​ TRUE;
 }

The actual code for the importer is almost laughably simple. We’re just loading the metadata file back into an NSDictionary, looping over the keys using the allKeys method, and adding each associated value to the passed-in CFMutableDictionaryRef. Once we’re done with the NSDictionary, we return TRUE and we’re done. Since we’re running inside a C function, we need to wrap the entire procedure in an NSAutoreleasePool so that we aren’t leaking any memory.

Testing the Spotlight Importer

There are a couple of ways to test the importer to make sure everything is working properly. The first thing we need to do is generate the metadata files, which we accomplish by running our application. Once the metadata files are created, we can test the importer.

We can get a lot of information about our importer directly on the command line. Mac OS X includes a command-line tool called mdimport. A quick review of the man page reveals there are three switches for this command that are of immediate use. First, we need to tell Spotlight to load our importer.

 mdimport -r ${path to our project}/build/Debug/GrokkingRecipes.app/
  Contents/Library/Spotlight/SpotlightPlugin.mdimporter

Once Spotlight is aware of our importer, we can start querying it, again from the command line using the mdimport command.

 cd ~/Library/Caches/Metadata/GrokkingRecipes
 mdimport -d2 Test.grokkingrecipe

We can change the debug level (from 1 to 4) to display different quantities of information about the metadata file. If we use level 2 we can confirm the importer is working and get a basic summary of the data contained inside the file.

The other way to test the importer is to just search for one of our recipes! Click the spotlight magnifying glass in the upper-right corner, and enter the name of one of the recipes, as shown here.

images/PotRoastInSpotlight.png

But what happens when we try to open this file?

Accepting Metadata Files

Since we linked our metadata files to the primary application, Mac OS X attempts to open our application and pass the file to us. However, we have no way of handling that yet. We need to teach our application to accept a request to open a file:

Spotlight/AppDelegate.m
 -​ (BOOL)application:(NSApplication*)theApplication
  openFile:(NSString*)filename
 {
  NSDictionary *meta = [NSDictionary dictionaryWithContentsOfFile:filename];
  NSString *objectIDString = [meta valueForKey:(id)kPPObjectID];
  NSURL *objectURI = [NSURL URLWithString:objectIDString];
  NSPersistentStoreCoordinator *coordinator;
  coordinator = [[self managedObjectContext] persistentStoreCoordinator];
 
  NSManagedObjectID *oID;
  oID = [coordinator managedObjectIDForURIRepresentation:objectURI];
 
  NSManagedObject *recipe = [[self managedObjectContext] objectWithID:oID];
 if​ (!recipe) ​return​ NO;
 
  dispatch_async(dispatch_get_main_queue(), ^{
  NSArray *array = [NSArray arrayWithObject:recipe];
  [[self recipeArrayController] setSelectedObjects:array];
  });
 
 return​ YES;
 }

In our application delegate, we need to add the method ‑(BOOL)application:openFile: that will be called when the operating system attempts to open one of our metadata files. In that method, we load the metadata file into an NSDictionary and retrieve the URIRepresentation of the NSManagedObjectID. With the NSManagedObjectID in hand, we can load the represented Recipe entity and display it to the user. Since we want to return from this method as quickly as possible (the operating system is waiting on an answer), we display the recipe after we return from this method.

To do that, we wrap the call to display the recipe in a dispatch_async, which updates the recipeArrayController with the selected recipe and allows the UI to update. By doing a dispatch_async and putting the execution block onto the main queue, we’re effectively telling the OS to run the block right after the current run loop completes.

With that code in place, we can select a recipe from Spotlight, and our application opens with the correct recipe selected. The first part of our OS integration is now in place.