How optimize code with introspection + heavy alloc on iPhone

I have a problem. I try to display a UITable that could have 2000-20000 records (typicall numbers.)

I have a SQLite database similar to the Apple contacts application.

I do all the tricks I know to get a smoth scroll, but I have a problem.

I load the data in 50 recods blocks. Then, when the user scroll, request next 50 until finish the list.

However, load that 50 records cause a notable "pause" in loading and scrolling. Everything else works fine.

I cache the data, have opaque cells, draw it by code, etc...

I swap the code loading the same data in dicts and have a performance boost but wonder if I could keep my object oriented aproach and improve the actual code.

This is the code I think have the performance problem:

    -(NSArray *) loadAndFill: (NSString *)sql theClass: (Class)cls {
        [self openDb];

        NSMutableArray *list = [NSMutableArray array];

        NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

        DbObject *ds;
        Class myClass = NSClassFromString([DbObject getTableName:cls]);

        FMResultSet *rs = [self load:sql];

        while ([rs next]) {
            ds = [[myClass alloc] init];
            NSDictionary *props = [ds properties];
            NSString *fieldType = nil;
            id fieldValue;

            for (NSString *fieldName in [props allKeys]) {
                fieldType = [props objectForKey: fieldName];

                fieldValue = [self ValueForField:rs Name:fieldName Type:fieldType];

                [ds setValue:fieldValue forKey:fieldName];
            }

            [list addObject :ds];

            [ds release];
        }
        [rs close];

        [pool drain];
        return list;
    }

And I think the main culprit is:

    -(id) ValueForField: (FMResultSet *)rs Name:(NSString *)fieldName Type:(NSString *)fieldType {
        id fieldValue = nil;

        if ([fieldType isEqualToString:@"i"] || // int
                 [fieldType isEqualToString:@"I"] || // unsigned int
                 [fieldType isEqualToString:@"s"] || // short
                 [fieldType isEqualToString:@"S"] || // unsigned short
                 [fieldType isEqualToString:@"f"] || // float
                 [fieldType isEqualToString:@"d"] )  // double
        {
            fieldValue = [NSNumber numberWithInt: [rs longForColumn:fieldName]];
        }
        else if ([fieldType isEqualToString:@"B"]) // bool or _Bool
        {
            fieldValue = [NSNumber numberWithBool: [rs boolForColumn:fieldName]];
        }
        else if ([fieldType isEqualToString:@"l"] || // long
                 [fieldType isEqualToString:@"L"] || // usigned long
                 [fieldType isEqualToString:@"q"] || // long long
                 [fieldType isEqualToString:@"Q"] ) // unsigned long long
        {
            fieldValue = [NSNumber numberWithLong: [rs longForColumn:fieldName]];
        }
        else if ([fieldType isEqualToString:@"c"] || // char
                 [fieldType isEqualToString:@"C"] ) // unsigned char

        {
            fieldValue = [rs stringForColumn:fieldName];
            //Is really a boolean?
            if ([fieldValue isEqualToString:@"0"] || [fieldValue isEqualToString:@"1"]) {
                fieldValue = [NSNumber numberWithInt: [fieldValue intValue]];
            }
        }
        else if ([fieldType hasPrefix:@"@"] ) // Object
        {
            NSString *className = [fieldType substringWithRange:NSMakeRange(2, [fieldType length]-3)];

            if ([className isEqualToString:@"NSString"]) {
                fieldValue = [rs stringForColumn:fieldName];
            }
            else if ([className isEqualToString:@"NSDate"]) {
                NSDateFormatter* dateFormatter = [[NSDateFormatter alloc] init];
                [dateFormatter setDateFormat:@"yyyy-MM-dd'T'HH:mm:ss"];
                NSString *theDate = [rs stringForColumn:fieldName];

                if (theDate) {
                    fieldValue = [dateFormatter dateFromString: theDate];
                }
                else
                {
                    fieldValue = nil;
                }

                [dateFormatter release];
            }
            else if ([className isEqualToString:@"NSInteger"]) {
                fieldValue = [NSNumber numberWithInt: [rs intForColumn :fieldName]];
            }
            else if ([className isEqualToString:@"NSDecimalNumber"]) {
                fieldValue = [rs stringForColumn :fieldName];
                if (fieldValue) {
                    fieldValue = [NSDecimalNumber decimalNumberWithString:[rs stringForColumn :fieldName]];
                }
            }
            else if ([className isEqualToString:@"NSNumber"]) {
                fieldValue = [NSNumber numberWithDouble: [rs doubleForColumn:fieldName]];
            }
            else
            {
                //Is a relationship one-to-one?
                if (![fieldType hasPrefix:@"NS"]) {
                    id rel =  class_createInstance(NSClassFromString(className), sizeof(unsigned));
                    Class theClass = [rel class];
                    if ([rel isKindOfClass:[DbObject class]]) {
                        fieldValue = [rel init];
                        //Load the record...
                        NSInteger Id = [rs intForColumn:[theClass relationName]];
                        if (Id>0) {
                            [fieldValue release];

                            Db *db = [Db currentDb];

                            fieldValue = [db loadById: theClass theId:Id];
                        }
                    }
                } else {

                    NSString *error = [NSString stringWithFormat:@"Err Can't get value for field %@ of type %@", fieldName, fieldType];

                    NSLog(error);
                    NSException *e = [NSException
                                      exceptionWithName:@"DBError"
                                      reason:error
                                      userInfo:nil];
                    @throw e;
                }
            }
        }
        return fieldValue;
    }

From stackoverflow

• You should be able to run this program with "Instruments" using the sampler tool to figure out where the problem is. You can rewrite ValueForField into smaller calls to see which part of it is the bottleneck, if necc.

  Other possibilities: If the data is static, you could load much (all of it?) at once into C arrays (especially for the ints and bools). If there are lots of values that are the same, you can share objects for the alike ones - eg: if the table has 20,000 lines, and 18,000 have the same string for some column, you can just make one string and share it.

• Look like the performance problem is with decimalNumberWithString.

  If I remove that from the code, the delay is a lot smaller than with it.

  Is bad that I need NSDecimalNumber for currency managment :(

  Crashworks : But do you really need it? Could you not store, say, a 64-bit fixed point number representing pennies (or centavos or whatever)? That would be more than enough to represent the entire world GDP, assuming you do not care about fractional pennies.