Proportional Scaling with NSImage
I know this will come as a shock, but not all images are square. Nor are all drawing canvases. In these cases, you may want to scale an image without getting it all stretchy. Here's a bit of code to help.This is already part of the SimplePicture project, but I thought it would be helpful to separate it out for those who don't need the rest of the application. Basically, we find the longest edge, get a scale factor, and apply it to both sides.
Like all code, it could be improved upon, but it should be enough for at least 80% of the common cases. I stripped all of the comments out for the inline version here in the interest of space. You can download the category file to get all the details.
If you have suggestions, feel free to share them. This was a pretty quick solution.
@implementation NSImage (ProportionalScaling)
- (NSImage*)imageByScalingProportionallyToSize:(NSSize)targetSize
{
NSImage* sourceImage = self;
NSImage* newImage = nil;
if ([sourceImage isValid])
{
NSSize imageSize = [sourceImage size];
float width = imageSize.width;
float height = imageSize.height;
float targetWidth = targetSize.width;
float targetHeight = targetSize.height;
float scaleFactor = 0.0;
float scaledWidth = targetWidth;
float scaledHeight = targetHeight;
NSPoint thumbnailPoint = NSZeroPoint;
if ( NSEqualSizes( imageSize, targetSize ) == NO )
{
float widthFactor = targetWidth / width;
float heightFactor = targetHeight / height;
if ( widthFactor < heightFactor )
scaleFactor = widthFactor;
else
scaleFactor = heightFactor;
scaledWidth = width * scaleFactor;
scaledHeight = height * scaleFactor;
if ( widthFactor < heightFactor )
thumbnailPoint.y = (targetHeight - scaledHeight) * 0.5;
else if ( widthFactor > heightFactor )
thumbnailPoint.x = (targetWidth - scaledWidth) * 0.5;
}
newImage = [[NSImage alloc] initWithSize:targetSize];
[newImage lockFocus];
NSRect thumbnailRect;
thumbnailRect.origin = thumbnailPoint;
thumbnailRect.size.width = scaledWidth;
thumbnailRect.size.height = scaledHeight;
[sourceImage drawInRect: thumbnailRect
fromRect: NSZeroRect
operation: NSCompositeSourceOver
fraction: 1.0];
[newImage unlockFocus];
}
return [newImage autorelease];
}
@end
Proportional Scaling with NSImage
Posted Sep 30, 2007 — 18 comments below
Posted Sep 30, 2007 — 18 comments below
Pieter Omvlee — Sep 30, 07 4676
I think it's preferable; lockFocus is really, really slow.
When you do use lockFocus (instead of setSize:) it might be a good idea to use the antiAlias and imageInterpolation methods from NSGraphicsContext.
(I'm sure you know this, but maybe somebody else doesn't)
Pieter
Blain — Sep 30, 07 4677
NSRect scaleSizeProportionallyInRect(NSSize origionalSize, NSRect destinationRect);
In which case it'd also offset using destinationRect.origin
Given that the next step after -imageByScaling... would most likely be to draw this new image in another context, you'd save an entire drawing round by moving the images in directly. In the case of sending values as a DataSource, you could just use a makeSize, and let the table do the centering work.
Seperating out the NSSize/NSRect code also has the added utility in case you want to center&scale or otherwise fit in text or resize controls.
Still, very useful.
Peter Hosey — Sep 30, 07 4678
Exercise for the reader: Do this using the Lanczos Transform filter in Core Image. Programmatically.
It's not nearly as hard as I just made it sound. ☺
Peter Hosey — Sep 30, 07 4679
Blain: You might also add a BOOL parameter to indicate whether the resize should be letterboxed (largest new size that fits entirely within the rectangle) or clipped (smallest new size that shares at least one of the size members of the destination rectangle).
This is the same choice as the letterboxed vs clipped option on the iPhone and iPod touch when you're watching a widescreen video.
Scott Stevenson — Sep 30, 07 4680
Scott Stevenson — Sep 30, 07 4681
In the context where this was originally conceived, the idea was to take very, very large images and resize them into 64x64 pixel thumbnails. In that case, you absolutely want to resample so you're not pulling all that extra image data around.
In any case, locking focus is not that slow if you use it in the way it's likely intended. It's probably best to not do it 30 times per second, but if you're caching an image (which is what happens in SimplePicture), it's absolutely fine.
@Blain: I was about to say, you could probably make things even faster by abstracting out the nsimage, and making a C-style function (...) Given that the next step after -imageByScaling... would most likely be to draw this new image in another context
That's a completely valid approach. In fact, you could use that sort of utility function inside of this one. But the point of this particular method is to make a resized/cached copy of the original image for later use.
@Peter Hosey: Do this using the Lanczos Transform filter in Core Image. Programmatically
Admittedly, I either didn't know or forgot about this, but there are some cases where you need to NSImage instead of CIImage (trust me! :). And other cases where it's just more convenient.
Still, thanks for bringing it up.
John C. Randolph — Sep 30, 07 4682
NSPoint thumbnailPoint = NSMakePoint(0,0);
to:
NSPoint thumbnailPoint = NSZeroPoint;
-jcr
Scott Stevenson — Sep 30, 07 4683
NSPoint thumbnailPoint = NSMakePoint(0,0)
True. Fixed inline. I'll fix it in the download at some point, as well. Thanks.
For the sake of accuracy, though, it's worth reminding everyone else that NSMakePoint is a static inline, so there's no function call per se.
Peter Hosey — Oct 01, 07 4684
@Scott Stevenson:
In such cases, you can use NSCIImageRep to wrap the CIImage inside an NSImage. ☺
Pieter Omvlee — Oct 01, 07 4685
Well, that's true. I was just wondering if, apart from the extra image data, there would be a difference in quality, but probably not.
Blain — Oct 01, 07 4687
I imagine you'd need to keep the aspect ratio, and therefore, recalculate the centering, but then the caching could be done by cocoa almost for free; Thumbnails would fall to the smaller imageRep, it'd be released when the image is released, and if you ever choose to do an iPhoto-like "Zoom from thumbnail to full size" slider, you don't have to worry about swapping in and out the cached version.
I'm still not seeing any goofball mistakes. What was it?
Scott Stevenson — Oct 01, 07 4688
That doesn't work in cases where you want to specifically avoid using CoreImage, which does happen. But again, it's good to mention.
@Blain: Is there any advantage or disadvantage to adding an ImageRep instead of generating a new image that I'm not seeing?
I admit I'm not quite sure what you mean. The goal in the case of SimplePicture was to open the full-size image, resample it as a thumbnail, then release the full size version to save memory.
You could draw the thumbnail into an NSBitmapImageRep, add that to the original image, then remove the full-size NSBitmapImageRep, but I'm not sure you'd end up ahead.
There are certainly cases where you want both small and large versions of the same source image, but I think you're better off just making separate NSImages in that cases, rather than one image with multiple bitmap reps. I think it would up being a lot of work to manage the reps since they're not keyed by size or anything. You'd have to enumerate through them each time to find the right one. The whole idea behind the NSImage/NSImageRep relationship is to encapsulate the details when possible.
Typically, I think you'd want multiple reps if you need different classes of reps. That is, an NSBitmapImageRep, an NSCIImageRep, and so on inside of one NSImage. I'm not so sure you'd want multiple instances of the same kind.
You do bring up an interesting point, though. If you add a NSBitmapImageRep of 500x500, and another at 64x64, will NSImage be smart enough to use the 64x64 version (the bitmap rep, not the cache rep) if you draw at that size? Worth an experiment.
I'm still not seeing any goofball mistakes. What was it?
Nothing to see here.
Steven — Oct 01, 07 4691
This is much easier (safer, always works...) than rolling your own (what I did). I never considered rendering it into another NSImage.
Steven
Blain — Oct 01, 07 4692
If you add a NSBitmapImageRep of 500x500, and another at 64x64, will NSImage be smart enough to use the 64x64 version (the bitmap rep, not the cache rep) if you draw at that size?
Perfect timing for this question. Historian is using the carbon IconRefs for grabbing file icons (to allow for Finder-style hilighting and badges), and the code I used (I think it's from Uli, but I'll have to double-check) makes a single NSImage with an ImageRep for each size of icon that the IconRef has. I think that -iconForFileType: does the same. Point being that the 16*16 icon is visually different than a scaled down 128*128, so when I played around with image sizes, I could tell the difference.
If I recall, Cocoa does the Right Thing in handling, finding the closest match that's larger, then the closest match that's smaller. I'll have to double-check that. Point being, if you do have to deal with constantly scaling images, a large thumbnail will still be a significant speedup.
BrentC — Nov 07, 07 5017
Thanks for a very useful column for a newbie to the Cocoa/ObjC world...
I think I may have found a problem with your -imageByScalingProportionallyToSize implementation, though, at least as it is used in the SimplePicture sample that I was playing with.
The issue shows itself when you browse a directory with very large pics... in my case, a directory with 114 8 MPixel images from my camera. What I saw initially was the SimplePic process immediately max out on virtual memory, and grind the system to a halt while processing the directory.
My initial investigations didn't find any memory leaks... what I did find (using the ObjAlloc tool), though, was a huge amount of memory being allocated in the call to NSImage:isValid.
On a hunch (being new to this ObjC stuff...) I thought that there might be a bunch of memory being autoreleased in that hidden OS code... and since 114 huge pictures are loaded in between auto-release pool releases, it seems that the auto-released (but not actually released) image buffers clog the system.
So, in NSImage-Extras.m, I added:
- (NSImage*)imageByScalingProportionallyToSize:(NSSize)targetSize { NSImage* sourceImage = self; NSImage* newImage = nil; NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
and then:
// remember to release our temporary auto-pool [pool release]; // And then add the image to the auto-release pool of the caller... return [newImage autorelease]; } @end
Is this the 'correct' way to handle this in ObjC? It seems to solve the problem, as memory usage is now correct, and any size directory now parses (in time...). But is there a better way?
Thanks again for a very useful set of columns.
Scott Stevenson — Nov 07, 07 5018
Your code looks correct for what you were trying to do. I'm actually really surprised this worked, but I guess there's more autorelease going on in -isValid than I would have guessed. In any case, you could probably just comment out the -isValid check without any real risk.
The real solution is for the application to change to be smarter about doing the images in batches. Ideally, it probably wouldn't even load actual image data until you started to scroll to the places where you might need it. But it wasn't really designed for scalability -- just a simple demo.
The new ImageKit framework in Leopard makes a lot of this vastly simpler.
BrentC — Nov 07, 07 5020
Thanks for a very useful column for a newbie to the Cocoa/ObjC world...
I think I may have found a problem with your -imageByScalingProportionallyToSize implementation, though, at least as it is used in the SimplePicture sample that I was playing with.
The issue shows itself when you browse a directory with very large pics... in my case, a directory with 114 8 MPixel images from my camera. What I saw initially was the SimplePic process immediately max out on virtual memory, and grind the system to a halt while processing the directory.
My initial investigations didn't find any memory leaks... what I did find (using the ObjAlloc tool), though, was a huge amount of memory being allocated in the call to NSImage:isValid.
On a hunch (being new to this ObjC stuff...) I thought that there might be a bunch of memory being autoreleased in that hidden OS code... and since 114 huge pictures are loaded in between auto-release pool releases, it seems that the auto-released (but not actually released) image buffers clog the system.
So, in NSImage-Extras.m, I added:
- (NSImage*)imageByScalingProportionallyToSize:(NSSize)targetSize { NSImage* sourceImage = self; NSImage* newImage = nil; [b]NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];[/b]
and then:
[b]// remember to release our temporary auto-pool [pool release]; // And then add the image to the auto-release pool of the caller...[/b] return [newImage autorelease]; } @end
Is this the 'correct' way to handle this in ObjC? It seems to solve the problem, as memory usage is now correct, and any size directory now parses (in time...). But is there a better way?
Thanks again for a very useful set of columns.
Brian Christensen — Jan 27, 08 5402