Simperium Updates "Freezing" device until finished -

hi simperium developers.

i building ios app using data platform, go deeper final testing see issue amount of coredata increases app "freezes" update.

the issue apparent when user has been using ipad version of app moves device , syncing large dataset first time new device, see similar behaviour small changes too.

so see following in log

simperium enqueuing 253 object requests (peoplemodel)

simperium enqueuing 301 object requests (seatsmodel)

simperium enqueuing 139 object requests (keywordmodel)

etc.

i have approx 20 entities in total

then app appears hang user interface until see

simperium finished processing index peoplemodel

simperium finished processing index seatsmodel

simperium finished processing index keywordmodel

there can many minutes of wait between enqueuing , finsheed processing messages.

if turn on verbose logging see additional information each object enqueued - no errors or warnings...

is there can do/change in ios app release rest of app continue processing ...?

help !

cheers

steve

hi mike,

thanks quick reply, adding usewebsockets appears make things better. on smaller updates helped. when main sync (i.e. scenario user syncing dataset exists in cloud , example when link new device , data exists) see same "freezing" behavior , received crash :

simperium websocket failed error error domain=org.lolrus.socketrocket code=2145 "error writing stream" userinfo=0x28b85f60 {nslocalizeddescription=error writing stream} 2013-05-13 19:29:56.898 meetingpad[652:907] * -[srwebsocket send:]: message sent deallocated instance 0x1d172bc0

i tried test 3 more times didn't see websocket error above again... possibly not relevant.

my coredata 2.4mb in size - here full console output, more thoughts appreciated.... happy popup message user saying "syncing please wait" i'm unsure how know when sync has finished remove message.....

2013-05-13 20:09:37.367 meetingpad[897:907] init dropbox 2013-05-13 20:09:37.575 meetingpad[897:907] init mpiaphelper 2013-05-13 20:09:37.588 meetingpad[897:1803] network reachable! 2013-05-13 20:09:37.676 meetingpad[897:907] simperium error: bucket list not loaded. ensure simperium started before objects fetched. 2013-05-13 20:09:37.702 meetingpad[897:907] simperium error: bucket list not loaded. ensure simperium started before objects fetched. 2013-05-13 20:09:37:928 meetingpad[897:907] simperium didn't find existing auth token 2013-05-13 20:09:37:956 meetingpad[897:907] simperium starting...  2013-05-13 20:09:37:960 meetingpad[897:907] simperium loaded 21 entity definitions 2013-05-13 20:09:37.977 meetingpad[897:907] simperium managing 0 actionlinks82 object instances 2013-05-13 20:09:38.000 meetingpad[897:907] simperium managing 1 relationship1model82 object instances 2013-05-13 20:09:38.004 meetingpad[897:907] simperium managing 0 attendeemodel82 object instances 2013-05-13 20:09:38.010 meetingpad[897:907] simperium managing 0 clipmodel82 object instances 2013-05-13 20:09:38.015 meetingpad[897:907] simperium managing 0 shapesmodel82 object instances 2013-05-13 20:09:38.020 meetingpad[897:907] simperium managing 0 seatmodel82 object instances 2013-05-13 20:09:38.026 meetingpad[897:907] simperium managing 0 agendaitemmodel82 object instances 2013-05-13 20:09:38.031 meetingpad[897:907] simperium managing 0 pointsmodel82 object instances 2013-05-13 20:09:38.036 meetingpad[897:907] simperium managing 0 agendaitemversionmodel82 object instances 2013-05-13 20:09:38.040 meetingpad[897:907] simperium managing 0 relationship2model82 object instances 2013-05-13 20:09:38.044 meetingpad[897:907] simperium managing 0 imagesmodel82 object instances 2013-05-13 20:09:38.054 meetingpad[897:907] simperium managing 1 seatingplanmodel82 object instances 2013-05-13 20:09:38.075 meetingpad[897:907] simperium managing 0 notelink82 object instances 2013-05-13 20:09:38.080 meetingpad[897:907] simperium managing 0 recordingmodel82 object instances 2013-05-13 20:09:38.085 meetingpad[897:907] simperium managing 0 actionsmodel82 object instances 2013-05-13 20:09:38.089 meetingpad[897:907] simperium managing 0 keywordlinks82 object instances 2013-05-13 20:09:38.093 meetingpad[897:907] simperium managing 0 peoplelinks82 object instances 2013-05-13 20:09:38.098 meetingpad[897:907] simperium managing 0 evernotedeletions82 object instances 2013-05-13 20:09:38.105 meetingpad[897:907] simperium managing 6 stylesmodel82 object instances 2013-05-13 20:09:38.107 meetingpad[897:907] simperium managing 1 notesmodel82 object instances 2013-05-13 20:09:38.111 meetingpad[897:907] simperium managing 0 peoplemodel82 object instances 2013-05-13 20:09:38:116 meetingpad[897:907] simperium didn't find existing auth token 2013-05-13 20:09:54:931 meetingpad[897:907] simperium authenticating: https://auth.simperium.com/1/wqqewweqeqw-disabeqweqwilities-33we2/authorize/ 2013-05-13 20:09:56:010 meetingpad[897:907] simperium authentication success! 2013-05-13 20:09:56.150 meetingpad[897:907] reachability flag status: -r ------- networkstatusforflags 2013-05-13 20:09:56:151 meetingpad[897:907] simperium starting network managers... 2013-05-13 20:09:56.151 meetingpad[897:907] opening connection... 2013-05-13 20:09:57:855 meetingpad[897:907] simperium processing 106 objects index (clipmodel82) 2013-05-13 20:09:57:878 meetingpad[897:907] simperium enqueuing 106 object requests (clipmodel82) 2013-05-13 20:09:58:069 meetingpad[897:907] simperium processing 290 objects index (attendeemodel82) 2013-05-13 20:09:58:124 meetingpad[897:907] simperium enqueuing 290 object requests (attendeemodel82) 2013-05-13 20:09:58:173 meetingpad[897:907] simperium processing 223 objects index (shapesmodel82) 2013-05-13 20:09:58:175 meetingpad[897:907] simperium processing 28 objects index (imagesmodel82) 2013-05-13 20:09:58:176 meetingpad[897:907] simperium processing 47 objects index (relationship2model82) 2013-05-13 20:09:58:196 meetingpad[897:907] simperium enqueuing 47 object requests (relationship2model82) 2013-05-13 20:09:58:205 meetingpad[897:907] simperium enqueuing 28 object requests (imagesmodel82) 2013-05-13 20:09:58:232 meetingpad[897:907] simperium processing 100 objects index (relationship1model82) 2013-05-13 20:09:58:234 meetingpad[897:907] simperium enqueuing 223 object requests (shapesmodel82) 2013-05-13 20:09:58:256 meetingpad[897:907] simperium enqueuing 100 object requests (relationship1model82) 2013-05-13 20:09:58:276 meetingpad[897:907] simperium processing 250 objects index (actionlinks82) 2013-05-13 20:09:58:322 meetingpad[897:907] simperium processing 251 objects index (seatingplanmodel82) 2013-05-13 20:09:58:348 meetingpad[897:907] simperium enqueuing 250 object requests (actionlinks82) 2013-05-13 20:09:58:378 meetingpad[897:907] simperium enqueuing 251 object requests (seatingplanmodel82) 2013-05-13 20:09:58:383 meetingpad[897:907] simperium processing 155 objects index (recordingmodel82) 2013-05-13 20:09:58:412 meetingpad[897:907] simperium enqueuing 155 object requests (recordingmodel82) 2013-05-13 20:09:58:442 meetingpad[897:907] simperium processing 24 objects index (stylesmodel82) 2013-05-13 20:09:58:449 meetingpad[897:907] simperium enqueuing 24 object requests (stylesmodel82) 2013-05-13 20:09:58:471 meetingpad[897:907] simperium processing 232 objects index (notelink82) 2013-05-13 20:09:58:482 meetingpad[897:907] simperium processing 289 objects index (actionsmodel82) 2013-05-13 20:09:58:486 meetingpad[897:907] simperium processing 248 objects index (notesmodel82) 2013-05-13 20:09:58:500 meetingpad[897:907] simperium processing 295 objects index (peoplelinks82) 2013-05-13 20:09:58:593 meetingpad[897:907] simperium enqueuing 232 object requests (notelink82) 2013-05-13 20:09:58:655 meetingpad[897:907] simperium enqueuing 248 object requests (notesmodel82) 2013-05-13 20:09:58:686 meetingpad[897:907] simperium enqueuing 295 object requests (peoplelinks82) 2013-05-13 20:09:58:696 meetingpad[897:907] simperium enqueuing 289 object requests (actionsmodel82) 2013-05-13 20:09:59:987 meetingpad[897:907] simperium processing 481 objects index (pointsmodel82) 2013-05-13 20:10:00:073 meetingpad[897:907] simperium processing 275 objects index (seatmodel82) 2013-05-13 20:10:00:080 meetingpad[897:907] simperium processing 267 objects index (keywordlinks82) 2013-05-13 20:10:00:088 meetingpad[897:907] simperium processing 220 objects index (peoplemodel82) 2013-05-13 20:10:00:092 meetingpad[897:907] simperium enqueuing 481 object requests (pointsmodel82) 2013-05-13 20:10:00:450 meetingpad[897:907] simperium enqueuing 220 object requests (peoplemodel82) 2013-05-13 20:10:00:461 meetingpad[897:907] simperium enqueuing 267 object requests (keywordlinks82) 2013-05-13 20:10:00:461 meetingpad[897:907] simperium enqueuing 275 object requests (seatmodel82) 2013-05-13 20:13:29.671 meetingpad[897:907] opening connection... 2013-05-13 20:13:30:598 meetingpad[897:907] simperium processing 100 objects index (relationship1model82) 2013-05-13 20:13:30:706 meetingpad[897:907] simperium processing 250 objects index (actionlinks82) 2013-05-13 20:13:30:743 meetingpad[897:907] simperium processing 290 objects index (attendeemodel82) 2013-05-13 20:13:30:804 meetingpad[897:907] simperium processing 106 objects index (clipmodel82) 2013-05-13 20:13:30:806 meetingpad[897:907] simperium processing 47 objects index (relationship2model82) 2013-05-13 20:13:30:846 meetingpad[897:907] simperium processing 275 objects index (seatmodel82) 2013-05-13 20:13:30:858 meetingpad[897:907] simperium processing 28 objects index (imagesmodel82) 2013-05-13 20:13:30:870 meetingpad[897:907] simperium processing 155 objects index (recordingmodel82) 2013-05-13 20:13:30:948 meetingpad[897:907] simperium processing 223 objects index (shapesmodel82) 2013-05-13 20:13:30:955 meetingpad[897:907] simperium processing 267 objects index (keywordlinks82) 2013-05-13 20:13:30:977 meetingpad[897:907] simperium processing 481 objects index (pointsmodel82) 2013-05-13 20:13:31:011 meetingpad[897:907] simperium processing 232 objects index (notelink82) 2013-05-13 20:13:31:053 meetingpad[897:907] simperium processing 289 objects index (actionsmodel82) 2013-05-13 20:13:31:062 meetingpad[897:907] simperium processing 295 objects index (peoplelinks82) 2013-05-13 20:13:31:076 meetingpad[897:907] simperium processing 24 objects index (stylesmodel82) 2013-05-13 20:13:31:338 meetingpad[897:907] simperium processing 220 objects index (peoplemodel82) 2013-05-13 20:14:34.901 meetingpad[897:907] opening connection... 

the sheer amount of data syncing across large number of buckets exposed performance problems. these have been fixed this commit.

in particular:

1. nsnotifications added/changed objects being triggered aggressively while indexing. since people don't seem make use of these anyway (during indexing), they've been removed now. they're still fired when objects added/changed otherwise.

2. the storage of metadata tracks pending relationships has been moved nsuserdefaults metadata directly on nspersistentstore.

3. the resolution of pending relationships has been moved own gcd queue, since touches database potentially during indexing.