Convert an NSImage to CIImage

I realize that Core Image makes working with programmable GPUs drastically easier than it would be otherwise, but some things about it aren't so obvious. Most importantly, how do you get a CIImage from an NSImage? I thought this would be easy, but there are a few gotchas.

First, we need to get an actual valid CIImage object. This is easy enough if the file is on disk because you can just do something like:

NSURL   * url   = [NSURL fileURLWithPath: @"myfile.jpg"];
CIImage * image = [CIImage imageWithContentsOfURL: url];


There is no step three.

But what if your NSImage object is not directly from a file on disk? There's the first gotcha. Neither the CIImage or NSImage have a method which converts to one from the other. Doh. CIImage does have an +imageWithCGImage method, but CGImage is not toll-free bridged with NSImage.

By the way, just so we don't get too lost, here's a recap of what we're dealing with:

CIImage: Core Image object. Fancy effects, GPU accelerated.
CGImage: Core Graphics (Quartz) data structure.
NSImage: Cocoa Image object.


Confused yet? You have every right to be.

All you need to know is my goal here is to take an NSImage object, convert it to a CIImage, then display it on screen. (For a more complete description of these frameworks, check out the Cocoa Graphics Framework post).

In Search of a Solution

I thought I had found a quick solution from Dan Wood, but it seems like something got lost in translation because NSImage has no method called -bitmap. Close, but not quite there.

CGImage does have a -initWithBitmapImageRep: method, but how do we get an NSBitmapImageRep? Blake Seely has a post on NSImage which describes that part of the forumla:

NSImage * image    = [self currentImage];
NSData  * tiffData = [image TIFFRepresentation];
NSBitmapImageRep * bitmap;
bitmap = [NSBitmapImageRep imageRepWithData:tiffData];


This puts us in the home stretch:

NSImage * image    = [self currentImage];
NSData  * tiffData = [image TIFFRepresentation];
NSBitmapImageRep * bitmap;
bitmap = [NSBitmapImageRep imageRepWithData:tiffData];

CIImage * ciImage = [[CIImage alloc] initWithBitmapImageRep:bitmap];


Now we have the CIImage, and we just have to display it, right? That's easy enough:

[ciImage drawAtPoint: NSMakePoint (0,0)
            fromRect: NSMakeRect  (0,0,128,128)
           operation: NSCompositeSourceOver
            fraction: 1.0];


Great! Except, one minor issue. CIImage has no concept of "flipped" as NSImage does. That means if you draw your CIImage into a flipped view, you get one of these:

Upside-down iTunes Icon


Hmmmm. Fortunately, Andy Matuschak has the answer in his post on Polished Metal buttons. The answer, dear friend, is that you use a CIAffineTransform filter to flip the image before it's drawn in the view. So, a bit more code:

CIFilter *transform = [CIFilter filterWithName:@"CIAffineTransform"];
[transform setValue:ciImage forKey:@"inputImage"];

NSAffineTransform *affineTransform = [NSAffineTransform transform];
[affineTransform translateXBy:0 yBy:128];
[affineTransform scaleXBy:1 yBy:-1];
[transform setValue:affineTransform forKey:@"inputTransform"];

CIImage * result = [transform valueForKey:@"outputImage"];


The one minor catch here is that Cocoa automatically includes CIImage.h but not CIFilter.h, so you need to explictly include it at the top of the file:

#import <QuartzCore/CIFilter.h>

Final Solution

So when all is said and done, this is how you get an NSImage from an arbitrary source, convert it to a CIImage, flip it, then draw it into a view:

#import <QuartzCore/CIFilter.h>

// convert NSImage to bitmap
NSImage * myImage  = [self currentImage];
NSData  * tiffData = [myImage TIFFRepresentation];
NSBitmapImageRep * bitmap;
bitmap = [NSBitmapImageRep imageRepWithData:tiffData];

// create CIImage from bitmap
CIImage * ciImage = [[CIImage alloc] initWithBitmapImageRep:bitmap];

// create affine transform to flip CIImage
NSAffineTransform *affineTransform = [NSAffineTransform transform];
[affineTransform translateXBy:0 yBy:128];
[affineTransform scaleXBy:1 yBy:-1];

// create CIFilter with embedded affine transform
CIFilter *transform = [CIFilter filterWithName:@"CIAffineTransform"];
[transform setValue:ciImage forKey:@"inputImage"];
[transform setValue:affineTransform forKey:@"inputTransform"];

// get the new CIImage, flipped and ready to serve
CIImage * result = [transform valueForKey:@"outputImage"];

// draw to view
[result drawAtPoint: NSMakePoint ( 0,0 )
           fromRect: NSMakeRect  ( 0,0,128,128 )
          operation: NSCompositeSourceOver
           fraction: 1.0];

// cleanup
[ciImage release];


Here's our nice iTunes icon in full CIImage glory. What complete overkill for a ATI Radeon X1600.

iTunes Image Displayed Correctly


It does seem like quite a bit of work, though it's still better than writing GPU code. That said, file bugs to make this simpler. By the way, if anybody knows an easier way to this in Tiger, feel free to speak up.
Design Element
Convert an NSImage to CIImage
Posted Nov 14, 2006 — 10 comments below




 

Petteri Kamppuri — Nov 14, 06 2401

Actually, there's a much shorter (measuring lines of code) and "more supported" (meaning there's an API) way:

+(NSImage *) imageFromCIImage:(CIImage *)ciImage
{
NSImage *image = [[[NSImage alloc] initWithSize:NSMakeSize([ciImage extent].size.width, [ciImage extent].size.height)] autorelease];

[image addRepresentation:[NSCIImageRep imageRepWithCIImage:ciImage]];

return image;
}

Petteri Kamppuri — Nov 14, 06 2402

Oh yes, you wanted it the other way around... I should get eyes, or something. I'm embarrassed.

Qwerty Denzel — Nov 14, 06 2403

Here's slightly shorter:
NSData *tiffData = [image TIFFRepresentation]; CIImage *ciImage = [CIImage imageWithData:tiffData]; CGRect cgRect = [ciImage extent]; NSRect nsRect = NSMakeRect(cgRect.origin.x,\ cgRect.origin.y, cgRect.size.width, cgRect.size.height); if ([self isFlipped]) { CGAffineTransform transform; transform = CGAffineTransformMakeTranslation(0.0,cgRect.size.height); transform = CGAffineTransformScale(transform, 1.0, -1.0); ciImage = [ciImage imageByApplyingTransform:transform]; } [ciImage drawAtPoint:NSZeroPoint fromRect:nsRect operation:NSCompositeSourceOver fraction:1.0];

Paul F. — Nov 14, 06 2404

Here's my solution:

NSSize size = [image size];
[image lockFocus];

NSRect imageRect = NSMakeRect(0, 0, size.width, size.height);

NSBitmapImageRep* rep = [[NSBitmapImageRep alloc] initWithFocusedViewRect: imageRect];
[rep autorelease];
CIImage *bitmap = [[CIImage alloc] initWithBitmapImageRep: rep];
[bitmap autorelease];
[image unlockFocus];

George — Nov 14, 06 2407

Two thoughts:
If you know your NSImage contains an NSBitmapImageRep you can eliminate the TIFFRepresentation line with something like:
bitmap = [myImage bestRepresentationForDevice:nil];
or
bitmap = [[myImage representations] objectAtIndex:0];

Yes less general and I have no idea if TIFFRepresentation adds significant overhead.

Secondly, if you're not applying any other CIFilters, is there any reason to convert an NSImage to CIImage? I first assumed it was just the simplicity of your example, but now I'm wondering if there is some other reason you'd need a CIImage over an NSImage....

Scott Stevenson — Nov 14, 06 2410 Scotty the Leopard

Secondly, if you're not applying any other CIFilters, is there any reason to convert an NSImage to CIImage?

This example doesn't really use CIImage-specific code, but the goal here was more to tell people how to convert if they need to.

Jonathan Saggau — Nov 14, 06 2411

That *is* the point, I think... for the functionality that you can only get by having a CIImage (which I can't help but read as "C 2 mage). Filters are fun. Batteries included.

Alex Keresztes — Nov 14, 06 2412

I had to do this in the past and I did this by adding new categories to NSImage:
- (NSBitmapImageRep *)bitmap;
{
NSSize imgSize = [self size];
NSBitmapImageRep* bitmap = [NSBitmapImageRep alloc];

[self lockFocus];
[bitmap initWithFocusedViewRect:NSMakeRect(0.0, 0.0, imgSize.width, imgSize.height)];
[self unlockFocus];

return bitmap;
}

- (CIImage *)CIImage;
{
return [[CIImage alloc] initWithBitmapImageRep:[self bitmap]];
}

Seems like this is probably the solution that Dan Wood was referring to?

Qwerty Denzel — Nov 20, 06 2460

A bit late, but with my version, you can change the first two lines to these for a more efficient solution:

NSBitmapImageRep *bitmapRep = [image bestRepresentationForDevice:nil];
CIImage *ciImage = [[CIImage alloc] initWithBitmapImageRep:bitmapRep];

Of course, with the ciImage needing to be released at some later point.

Peter Hosey — Dec 03, 06 2553

“The one minor catch here is that Cocoa automatically includes CIImage.h but not CIFilter.h, so you need to explictly include it at the top of the file:”

Cocoa doesn't implicitly import anything from QuartzCore. It does currently, but it's not guaranteed to; it only imports CIImage so that it can add a category on that class. Relying on that implicit include is unclean at best, and at worst, it will cause compiler errors.

The correct way is to import QuartzCore.h:

#import <QuartzCore/QuartzCore.h>

That's the general rule on importing anything from a framework: Import the framework's overall header, which is named after the framework itself. Importing only specific headers invites preprocessor and compiler errors (error: FOO not defined). It may not happen the first 100 times you do it, but it will happen at some point. And this is Apple-defined behavior (http://developer.apple.com/documentation/MacOSX/Conceptual/BPFrameworks/Tasks/IncludingFrameworks.html).

The only Apple public framework for which this doesn't work is Message.framework. There is no <Message/Message.h>; you must import <Message/NSMailDelivery.h> directly. No idea why.

And before anybody complains about the performance cost of importing the whole framework: Set a prefix header and tell Xcode to precompile it, and then don't worry about it.




 

Comments Temporarily Disabled

I had to temporarily disable comments due to spam. I'll re-enable them soon.





Copyright © Scott Stevenson 2004-2015