Discussion:
CoreData headaches
(too old to reply)
Steve Mills
2016-10-27 21:24:05 UTC
Permalink
I have an app that uses CoreData to store document data, and an IKImageBrowserView to display the items in the CoreData. Finding that IKImageBrowserView is being deprecated, and since Xcode 8 causes a goofy scrolling bug that the user can't work around, I decided to move away from IKImageBrowserView and reimplement the display as an NSCollectionView. OK, that works. In order to match the speed at which IKImageBrowserView builds thumbnails in the background, I used an NSOperationQueue in which to load each thumbnail via CGImageSourceCreateThumbnailAtIndex. This mostly works fine, except I get random crashes when accessing properties of a NSManagedObject (the image file's folder's url and image file's name). Any suggestions for making this work correctly? I'm just hacking at it right now by adding @synchronize() wrappers around code, but it's not helping.

Since CoreData and managed objects are full of so much voodoo (and because it's been a long time since I originally wrote this app, so the CoreData stuff is pretty fuzzy by now), I wonder if there isn't a better system I could use to store document data. The app is an image organizer that can store references to thousands of images and movies (not the actual images), sort and filter them any number of ways (which is all handled by an NSArrayController), and tag them with keywords (also NSManagedObject).

What does CoreData give me that I can't get by simply archiving an array of NSDictionary? Well, aside from undo, writing only changed objects to the file, and all the rest of the stuff it does.

The previous version of the app used my own file format. I don't recall it being particularly slow or featureless. Hmm.

--
Steve Mills
Drummer, Mac geek


_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent to ***@ml-in
Dave Fernandes
2016-10-28 00:00:14 UTC
Permalink
Are you accessing the properties from within a NSManagedObjectContext.performBlock block? Sounds like you may be accessing the managed objects from the wrong queue.
Post by Steve Mills
Since CoreData and managed objects are full of so much voodoo (and because it's been a long time since I originally wrote this app, so the CoreData stuff is pretty fuzzy by now), I wonder if there isn't a better system I could use to store document data. The app is an image organizer that can store references to thousands of images and movies (not the actual images), sort and filter them any number of ways (which is all handled by an NSArrayController), and tag them with keywords (also NSManagedObject).
What does CoreData give me that I can't get by simply archiving an array of NSDictionary? Well, aside from undo, writing only changed objects to the file, and all the rest of the stuff it does.
I think you just answered your own question. The relationship management and delete rules are also helpful — there is less bookkeeping for you to do.
Post by Steve Mills
The previous version of the app used my own file format. I don't recall it being particularly slow or featureless. Hmm.
If you aren’t dealing with large numbers of objects, then all the optimizations in Core Data may not be important to you.
Post by Steve Mills
--
Steve Mills
Drummer, Mac geek
_______________________________________________
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
https://lists.apple.com/mailman/options/cocoa-dev/dave.fernandes%40utoronto.ca
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent to ***@ml-
Steve Mills
2016-10-28 03:24:18 UTC
Permalink
Post by Dave Fernandes
Are you accessing the properties from within a NSManagedObjectContext.performBlock block? Sounds like you may be accessing the managed objects from the wrong queue.
I'll explain what's going on. Each NSCollectionViewItem in the NSCollectionView is being assigned its representedObject (the Asset, which is a reference to an image file). At that point, I ask the Asset to requestPreviewImageAtSize:, which creates an NSBlockOperation (and adds it to an NSOperationQueue), whose block creates a CGImageSourceRef. Before it can create the CGImageSourceRef, it needs to get the url. It does this by asking for its .folder and .name properties. Those are nonatomic properties of the Asset. Once it has the url and the CGImageSourceRef, it creates the CGImageRef, converts that to an NSImage, and finally assigns that to the Asset's atomic .thumb property.

The folder and name properties were generated by mogenerator, a tool for creating document model classes for CoreData .xcdatamodeld files. The thumb property is something I added manually to cache the thumbnail, and the AssetItemView's NSImageView's Value binds to it as well.

So I'm not directly dealing with any NSManagedObjectContext here. That's all taken care of when the items are added to it or when a document is opened. All I'm doing is generating a preview for each managed object.

--
Steve Mills
Drummer, Mac geek


_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This
Dave Fernandes
2016-10-28 03:49:15 UTC
Permalink
But what managed object are you dealing with? Is the Asset a managed object? If so, you can only access its properties from the queue of its managed object context. So if it is a main queue context, you can only access the MO’s properties from the main queue. If it is a private queue context, you use performBlock on the MO’s context to execute a block of code in the context’s queue and you must access the MO’s properties from inside that block. If I’m stating something you already know, forgive me. It is not clear from your post. Example code would help.
Post by Steve Mills
Post by Dave Fernandes
Are you accessing the properties from within a NSManagedObjectContext.performBlock block? Sounds like you may be accessing the managed objects from the wrong queue.
I'll explain what's going on. Each NSCollectionViewItem in the NSCollectionView is being assigned its representedObject (the Asset, which is a reference to an image file). At that point, I ask the Asset to requestPreviewImageAtSize:, which creates an NSBlockOperation (and adds it to an NSOperationQueue), whose block creates a CGImageSourceRef. Before it can create the CGImageSourceRef, it needs to get the url. It does this by asking for its .folder and .name properties. Those are nonatomic properties of the Asset. Once it has the url and the CGImageSourceRef, it creates the CGImageRef, converts that to an NSImage, and finally assigns that to the Asset's atomic .thumb property.
The folder and name properties were generated by mogenerator, a tool for creating document model classes for CoreData .xcdatamodeld files. The thumb property is something I added manually to cache the thumbnail, and the AssetItemView's NSImageView's Value binds to it as well.
So I'm not directly dealing with any NSManagedObjectContext here. That's all taken care of when the items are added to it or when a document is opened. All I'm doing is generating a preview for each managed object.
--
Steve Mills
Drummer, Mac geek
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent to g
Steve Mills
2016-10-28 04:02:22 UTC
Permalink
Post by Dave Fernandes
But what managed object are you dealing with? Is the Asset a managed object? If so, you can only access its properties from the queue of its managed object context. So if it is a main queue context, you can only access the MO’s properties from the main queue. If it is a private queue context, you use performBlock on the MO’s context to execute a block of code in the context’s queue and you must access the MO’s properties from inside that block. If I’m stating something you already know, forgive me. It is not clear from your post. Example code would help.
Yes, the Asset is an NSManagedObject. In this call chain, there is no NSManagedObjectContext in sight. The NSCollectionView is simply creating item views to display for each Asset. It asks the Asset to create an image thumbnail. The Asset does so in via the steps I outlined before.

Now way back in the document's init method, I set up an NSManagedObjectContext with type NSMainQueueConcurrencyType. So yes, there is a main queue type context, but the code that is requesting the thumbnails knows nothing about it or even cares.
Post by Dave Fernandes
Post by Steve Mills
I'll explain what's going on. Each NSCollectionViewItem in the NSCollectionView is being assigned its representedObject (the Asset, which is a reference to an image file). At that point, I ask the Asset to requestPreviewImageAtSize:, which creates an NSBlockOperation (and adds it to an NSOperationQueue), whose block creates a CGImageSourceRef. Before it can create the CGImageSourceRef, it needs to get the url. It does this by asking for its .folder and .name properties. Those are nonatomic properties of the Asset. Once it has the url and the CGImageSourceRef, it creates the CGImageRef, converts that to an NSImage, and finally assigns that to the Asset's atomic .thumb property.
The folder and name properties were generated by mogenerator, a tool for creating document model classes for CoreData .xcdatamodeld files. The thumb property is something I added manually to cache the thumbnail, and the AssetItemView's NSImageView's Value binds to it as well.
So I'm not directly dealing with any NSManagedObjectContext here. That's all taken care of when the items are added to it or when a document is opened. All I'm doing is generating a preview for each managed object.
--
Steve Mills
Drummer, Mac geek


_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This
Dave Fernandes
2016-10-28 04:36:01 UTC
Permalink
The managed objects exist in a MOC whether you have a reference to that MOC or not. You can get a reference to the MOC that an MO “belongs to" from the -[NSManagedObject managedObjectContext] instance method. Since the properties you need are so few and simple, why don’t you just pass these in to the NSOperation when you create it on the main thread instead of giving it the managed object? Then the MO will never be accessed off the main queue.
Post by Steve Mills
Post by Dave Fernandes
But what managed object are you dealing with? Is the Asset a managed object? If so, you can only access its properties from the queue of its managed object context. So if it is a main queue context, you can only access the MO’s properties from the main queue. If it is a private queue context, you use performBlock on the MO’s context to execute a block of code in the context’s queue and you must access the MO’s properties from inside that block. If I’m stating something you already know, forgive me. It is not clear from your post. Example code would help.
Yes, the Asset is an NSManagedObject. In this call chain, there is no NSManagedObjectContext in sight. The NSCollectionView is simply creating item views to display for each Asset. It asks the Asset to create an image thumbnail. The Asset does so in via the steps I outlined before.
Now way back in the document's init method, I set up an NSManagedObjectContext with type NSMainQueueConcurrencyType. So yes, there is a main queue type context, but the code that is requesting the thumbnails knows nothing about it or even cares.
Post by Dave Fernandes
Post by Steve Mills
I'll explain what's going on. Each NSCollectionViewItem in the NSCollectionView is being assigned its representedObject (the Asset, which is a reference to an image file). At that point, I ask the Asset to requestPreviewImageAtSize:, which creates an NSBlockOperation (and adds it to an NSOperationQueue), whose block creates a CGImageSourceRef. Before it can create the CGImageSourceRef, it needs to get the url. It does this by asking for its .folder and .name properties. Those are nonatomic properties of the Asset. Once it has the url and the CGImageSourceRef, it creates the CGImageRef, converts that to an NSImage, and finally assigns that to the Asset's atomic .thumb property.
The folder and name properties were generated by mogenerator, a tool for creating document model classes for CoreData .xcdatamodeld files. The thumb property is something I added manually to cache the thumbnail, and the AssetItemView's NSImageView's Value binds to it as well.
So I'm not directly dealing with any NSManagedObjectContext here. That's all taken care of when the items are added to it or when a document is opened. All I'm doing is generating a preview for each managed object.
--
Steve Mills
Drummer, Mac geek
_______________________________________________
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
https://lists.apple.com/mailman/options/cocoa-dev/dave.fernandes%40utoronto.ca
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.na
Steve Mills
2016-10-29 04:44:27 UTC
Permalink
Post by Dave Fernandes
The managed objects exist in a MOC whether you have a reference to that MOC or not. You can get a reference to the MOC that an MO “belongs to" from the -[NSManagedObject managedObjectContext] instance method. Since the properties you need are so few and simple, why don’t you just pass these in to the NSOperation when you create it on the main thread instead of giving it the managed object? Then the MO will never be accessed off the main queue.
I moved the CGImageSource creation to outside the block, which is where I needed to access the managed object's folder and name properties. The block now just loads the image from the source, converts it to an NSImage, and sets the managed object's thumb property. It's no longer crashing, but the setting of the thumb property seems like that shouldn't happen inside the block. So would that be the right place to use the NSManagedObjectContext's performBlock:? Here's the managed object's method:

-(void) requestPreviewImageAtSize:(CGFloat)size
{
if(_thumbOp)
[_thumbOp cancel];

if([self createImageSource]) {
_thumbOp = [NSBlockOperation blockOperationWithBlock:^{
if(_thumbOp.isCancelled)
return;

NSDictionary* opts = @{(__bridge NSString*)kCGImageSourceCreateThumbnailFromImageAlways:@YES, (__bridge NSString*)kCGImageSourceThumbnailMaxPixelSize:@(size)};
CGImageRef thumbnail = CGImageSourceCreateThumbnailAtIndex(_imageSource, 0, (__bridge CFDictionaryRef)opts);

if(thumbnail) {
if(!_thumbOp.isCancelled) {
NSImage* image = [[NSImage alloc] initWithCGImage:thumbnail size:NSZeroSize];

if(image && !_thumbOp.isCancelled) {
[self.managedObjectContext performBlock:^{
self.thumb = image;
}];
}
}

CGImageRelease(thumbnail);
}
}];

[_thumbOp setCompletionBlock:^{
_thumbOp = nil;
}];

[[[self class] previewLoadingOperationQueue] addOperation:_thumbOp];
}
}

--
Steve Mills
Drummer, Mac geek


_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This
Chris Hanson
2016-10-30 19:16:45 UTC
Permalink
Post by Steve Mills
Post by Dave Fernandes
The managed objects exist in a MOC whether you have a reference to that MOC or not. You can get a reference to the MOC that an MO “belongs to" from the -[NSManagedObject managedObjectContext] instance method. Since the properties you need are so few and simple, why don’t you just pass these in to the NSOperation when you create it on the main thread instead of giving it the managed object? Then the MO will never be accessed off the main queue.
I moved the CGImageSource creation to outside the block, which is where I needed to access the managed object's folder and name properties. The block now just loads the image from the source, converts it to an NSImage, and sets the managed object's thumb property. It's no longer crashing, but the setting of the thumb property seems like that shouldn't happen inside the block.
You’re correct in this, reading a property of an NSManagedObject from another thread isn’t safe, and writing it isn’t either.
Post by Steve Mills
So would that be the right place to use the NSManagedObjectContext's performBlock:?
What you’ve done actually looks correct, in that you’re only interacting with the managed object’s managed (Core Data) properties (“thumb” in this case) within the block passed to -performBlock:.

-- Chris


_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.na
Chris Hanson
2016-10-30 19:11:01 UTC
Permalink
Post by Steve Mills
Yes, the Asset is an NSManagedObject. In this call chain, there is no NSManagedObjectContext in sight.
There is always an NSManagedObjectContext involved; an NSManagedObject doesn’t exist outside one. Fortunately, you don’t need to pass one along with an NSManagedObject, you can just ask the NSManagedObject for the context it’s a part of.

-- Chris

_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sen
Steve Mills
2016-10-31 03:22:02 UTC
Permalink
Post by Chris Hanson
Post by Steve Mills
Yes, the Asset is an NSManagedObject. In this call chain, there is no NSManagedObjectContext in sight.
There is always an NSManagedObjectContext involved; an NSManagedObject doesn’t exist outside one. Fortunately, you don’t need to pass one along with an NSManagedObject, you can just ask the NSManagedObject for the context it’s a part of.
That was my misunderstanding because of the confusion of the docs, or maybe the lack of generous explanation in the docs. The explanation they give sounds like they mean the context created specifically to do an operation, like the way that a temporary private context is created, say, to do a bulk add, then is merged with the parent main context when complete. That's why I said there wasn't a context in that call chain, meaning I didn't create a private context to do this work.

It's a bit more clear now, thanks to all of you that have chimed in here.

--
Steve Mills
Drummer, Mac geek


_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-i
Samuel Williams
2016-10-31 05:50:59 UTC
Permalink
Post by Steve Mills
CoreData [is] just a pile of confusion with no human explanation[.]
I know this doesn't help anyone but I couldn't help but agreeing with
this statement.
Post by Steve Mills
Post by Chris Hanson
Post by Steve Mills
Yes, the Asset is an NSManagedObject. In this call chain, there is no NSManagedObjectContext in sight.
There is always an NSManagedObjectContext involved; an NSManagedObject doesn’t exist outside one. Fortunately, you don’t need to pass one along with an NSManagedObject, you can just ask the NSManagedObject for the context it’s a part of.
That was my misunderstanding because of the confusion of the docs, or maybe the lack of generous explanation in the docs. The explanation they give sounds like they mean the context created specifically to do an operation, like the way that a temporary private context is created, say, to do a bulk add, then is merged with the parent main context when complete. That's why I said there wasn't a context in that call chain, meaning I didn't create a private context to do this work.
It's a bit more clear now, thanks to all of you that have chimed in here.
--
Steve Mills
Drummer, Mac geek
_______________________________________________
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
https://lists.apple.com/mailman/options/cocoa-dev/space.ship.traveller%40gmail.com
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.n
Jens Alfke
2016-10-28 18:37:37 UTC
Permalink
Post by Dave Fernandes
But what managed object are you dealing with? Is the Asset a managed object? If so, you can only access its properties from the queue of its managed object context. So if it is a main queue context, you can only access the MO’s properties from the main queue.
Amen, brother. I design/work on a storage framework with similar thread-safety rules (Couchbase Lite) and I can attest that quite a few Cocoa app developers seem to add concurrency to their code without giving it enough thought. I’ve helped people troubleshoot code where it seems like they just went “this operation is blocking the UI so I’ll just use dispatch_async to do it in the background”, without understanding the implications for race conditions, or following the thread-safety rules of the APIs they’re calling.

I’m not saying that you did this, Steve! Just using this thread as a reminder to people that Concurrency is Hard, and you have to think carefully about what you’re doing. If you aren’t very aware of what queue/thread every part of your code can run on, and what data might be accessible to other queues/threads, you’re likely to find yourself in a world of pain at some point.

—Jens
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent to geg
Steve Mills
2016-10-28 20:20:23 UTC
Permalink
On Oct 28, 2016, at 01:37 PM, Jens Alfke <***@mooseyard.com> wrote:


On Oct 27, 2016, at 8:49 PM, Dave Fernandes <***@utoronto.ca> wrote:
But what managed object are you dealing with? Is the Asset a managed object? If so, you can only access its properties from the queue of its managed object context. So if it is a main queue context, you can only access the MO’s properties from the main queue.

Amen, brother. I design/work on a storage framework with similar thread-safety rules (Couchbase Lite) and I can attest that quite a few Cocoa app developers seem to add concurrency to their code without giving it enough thought. I’ve helped people troubleshoot code where it seems like they just went “this operation is blocking the UI so I’ll just use dispatch_async to do it in the background”, without understanding the implications for race conditions, or following the thread-safety rules of the APIs they’re calling.

I’m not saying that you did this, Steve! Just using this thread as a reminder to people that Concurrency is Hard, and you have to think carefully about what you’re doing. If you aren’t very aware of what queue/thread every part of your code can run on, and what data might be accessible to other queues/threads, you’re likely to find yourself in a world of pain at some point.

No, I'm sure I *would* do something like that. :) To me, the CoreData docs are just a pile of confusion with no human explanation that I can understand. Maybe I should read it again to see if it's been improved recently. Or, more likely, I glossed over reading that part back when I started with CoreData because IKImageBrowserView took care of creating thumbnails for me, so crap just worked.

Sent from iCloud's ridiculous UI, so, sorry about the formatting

 
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent
Jens Alfke
2016-10-28 20:25:05 UTC
Permalink
Post by Steve Mills
No, I'm sure I *would* do something like that. :) To me, the CoreData docs are just a pile of confusion with no human explanation that I can understand.
Well, the general rule in a nutshell is that you should only use a CoreData object on the thread/queue that it was created on, unless the docs explicitly say otherwise.

—Jens
_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent to ***@ml-in.narkive.ne
Quincey Morris
2016-10-28 20:36:41 UTC
Permalink
Post by Steve Mills
the CoreData docs are just a pile of confusion with no human explanation that I can understand
CoreData [is] just a pile of confusion with no human explanation[.]
OTOH, I admit I was burned pretty badly by CoreData, so perhaps I’m a tiny bit biased. ;)

_______________________________________________

Cocoa-dev mailing list (Cocoa-***@lists.apple.com)

Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com

Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/cocoa-dev/gegs%40ml-in.narkive.net

This email sent to ***@ml-in.nar
Continue reading on narkive:
Loading...