I have functions to encrypt/decrypt using AES (128 and 256) on both iOS (CCCrypt) and python (pycryptdome). All test cases working on each platform but… when I take an AES key and encrypted string from iOS to python the decryption fails. I have looked extensively and tried various use cases to no avail.
I’ve created a simple test case here with an iOS encryption and python decryption in the hopes that someone can tell me what i am doing differently on the platforms.
iOS code Test Case
NSString *test_aes = @"XSmTe1Eyw8JsZkreIFUpNi7BhKEReHTP";
NSString *test_string = @"This is a test string";
NSData *clearPayload = [test_string dataUsingEncoding:NSUTF8StringEncoding];
NSData *encPayload = nil;
char keyPtr[kCCKeySizeAES256 + 1]; // room for terminator (unused)
bzero( keyPtr, sizeof( keyPtr ) ); // fill with zeroes (for padding)
// fetch key data
[test_aes getCString:keyPtr maxLength:sizeof( keyPtr ) encoding:NSUTF8StringEncoding];
NSUInteger dataLength = clearPayload.length;
size_t bufferSize = dataLength + kCCKeySizeAES256;
void *buffer = malloc( bufferSize );
size_t numBytesEncrypted = 0;
CCCryptorStatus cryptStatus = CCCrypt( kCCEncrypt, kCCAlgorithmAES, kCCOptionPKCS7Padding,
keyPtr, kCCKeySizeAES256,
NULL /* initialization vector (optional) */,
[clearPayload bytes], dataLength, /* input */
buffer, bufferSize, /* output */
&numBytesEncrypted );
NSString *encString = @"Error";
if( cryptStatus == kCCSuccess )
{
//the returned NSData takes ownership of the buffer and will free it on deallocation
encPayload = [NSData dataWithBytesNoCopy:buffer length:numBytesEncrypted];
encString = [encPayload base64EncodedStringWithOptions:NSDataBase64EncodingEndLineWithLineFeed];
}
//free( buffer ); //free the buffer
NSLog(@"Src = %@ AES = %@ String = %@",test_string, test_aes, encString);
encPayload = [[NSData alloc] initWithBase64EncodedString:encString options:NSDataBase64DecodingIgnoreUnknownCharacters];
clearPayload = nil;
char keyPtr2[kCCKeySizeAES256+1]; // room for terminator (unused)
bzero( keyPtr2, sizeof( keyPtr2 ) ); // fill with zeroes (for padding)
// fetch key data
[test_aes getCString:keyPtr2 maxLength:sizeof( keyPtr2 ) encoding:NSUTF8StringEncoding];
NSUInteger dataLength2 = [encPayload length];
//See the doc: For block ciphers, the output size will always be less than or
//equal to the input size plus the size of one block.
//That's why we need to add the size of one block here
size_t bufferSize2 = dataLength2 + kCCKeySizeAES256;
void *buffer2 = malloc( bufferSize2 );
size_t numBytesDecrypted = 0;
CCCryptorStatus cryptStatus2 = CCCrypt( kCCDecrypt, kCCAlgorithmAES, kCCOptionPKCS7Padding,
keyPtr, kCCKeySizeAES256,
NULL /* initialization vector (optional) */,
[encPayload bytes], dataLength2, /* input */
buffer2, bufferSize2, /* output */
&numBytesDecrypted );
NSString *clearString = @"Error";
if( cryptStatus2 == kCCSuccess )
{
//the returned NSData takes ownership of the buffer and will free it on deallocation
clearPayload = [NSData dataWithBytesNoCopy:buffer2 length:numBytesDecrypted];
clearString = [[NSString alloc] initWithData:clearPayload encoding:NSUTF8StringEncoding];
}
NSLog(@"Res = %@",clearString);
The encryption and decryption in this code works fine and the output is:
Src = This is a test string AES = XSmTe1Eyw8JsZkreIFUpNi7BhKEReHTP String = hUbjWyXX4mB01gI0RJhYQRD0iAjQnkGTpsnKcmDpvaQ= Res = This is a test string
When I take the encoded string and aes key to python to test with this code:
key = "XSmTe1Eyw8JsZkreIFUpNi7BhKEReHTP"
data = "hUbjWyXX4mB01gI0RJhYQRD0iAjQnkGTpsnKcmDpvaQ="
usekey = key
useData = data
if isinstance(key, str):
usekey = key.encode('utf-8')
cipher = AES.new(usekey, AES.MODE_GCM, nonce=self.nonce)
print("nonce", cipher.nonce)
if isinstance(data, str):
useData = data.encode('utf-8')
useData = b64decode(useData)
puseData = useData # unpad(useData,32)
print("decrypt:In bytes=", puseData)
result = cipher.decrypt(puseData)
print ("decrypt:Out bytes=",result)
The decryption fails with output of
nonce b'x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00x00'
decrypt:In bytes= b'x85Fxe3[%xd7xe2`txd6x024Dx98XAx10xf4x88x08xd0x9eAx93xa6xc9xcar`xe9xbdxa4'
decrypt:Out bytes= b'x08xc58x962qx94xff#xfaxabxe2xc8{bxedx0bxedwx8fxe3xecx0bx8exfbxccx12x7fx9exb4x8fxd6'
Both of the above routines work with locally encrypted data without issue, I have hacked the examples here (including not freeing the malloc’ed buffers :) for debugging purposes, so i apologize for the somewhat dirty code.
Note: I have tried changing the python mode to AES.MODE_CBC (and added padding code) when i saw notes iOS may use this rather than GCM, this failed as well… For now I have kept the nonce / iv as an array of 0 as I am told iOS will have used this as the default as the CCCrypt is not provided one, when this example works I will transition to specified iv.
I’d appreciate any direction.
EDIT: I went ahead and specified a null IV on the iOS side with
char iv[16]; // also tried 17
bzero( iv, sizeof( iv ) );
No change in behaviour at all…
EDIT: I set the IV to all char ‘1’ on both systems and got the same result. iOS added code:
NSString *hardCodeIV = @"1111111111111111";
char iv[17];
bzero( iv, sizeof( iv ) );
[hardCodeIV getCString:iv maxLength:sizeof(iv) encoding:NSUTF8StringEncoding];
Which produced
Src = This is a test string AES = XSmTe1Eyw8JsZkreIFUpNi7BhKEReHTP String = sFoZ24VRN1hyMzegXT+GFzAn/YGPvaKO8p1eD+xhGaU= Res = This is a test string
So on iOS it encrypts and decrypts properly with the byte 0 and char 1 IV….
And the python code works as well when encrypted and decrypted locally with either IV… But when the output from iOS encryption is used on python to decrypt it fails as shown here.
Moving the key and encrypted message to python for decryption as:
key = "XSmTe1Eyw8JsZkreIFUpNi7BhKEReHTP"
data = "sFoZ24VRN1hyMzegXT+GFzAn/YGPvaKO8p1eD+xhGaU="
usekey = key
useData = data
if isinstance(key, str):
usekey = key.encode('utf-8')
cipher = AES.new(usekey, AES.MODE_GCM, nonce=self.nonce)
print("nonce", cipher.nonce)
if isinstance(data, str):
useData = data.encode('utf-8')
useData = b64decode(useData)
puseData = useData # unpad(useData,32)
print("decrypt:In bytes=", puseData)
result = cipher.decrypt(puseData)
print ("decrypt:Out bytes=",result)
Resulted in:
nonce b'1111111111111111'
decrypt:In bytes= b"xb0Zx19xdbx85Q7Xr37xa0]?x86x170'xfdx81x8fxbdxa2x8exf2x9d^x0fxecax19xa5"
decrypt:Out bytes= b'xc3x1e"wx86:~x86xd3xc9H3xd3xd3y)|,|xe02(xc6x17xa3x1exe2x0fx1a#xbbW'
So, still no joy…
It looks very much like the algorithm choice is the problem, but the options on iOS seems to only be GCM or CBC with GCM being the default… Most testing has been done on GCM. I attempted to use CBC in one test (with no IV as it does not need one) in case iOS was actually using this and not telling me, but as shown above, that also had no success.
I’m continuing to test approaches, but could really use some advice from someone who has made this work – i have not been able to find working examples. [as a side note, the RSA models work fine – this is how i am moving the AES key around – and that part of the solution is flawless at the moment, this is the last bit i need to get operational).
Advertisement
Answer
Edited to final answer with both ECB and CBC working between iOS and Python:
With credit to others who built the origional NSData_AESCrypt code at:
// AES Encrypt/Decrypt // Created by Jim Dovey and 'Jean' // See http://iphonedevelopment.blogspot.com/2009/02/strong-encryption-for-cocoa-cocoa-touch.html // // BASE64 Encoding/Decoding // Copyright (c) 2001 Kyle Hammond. All rights reserved. // Original development by Dave Winer. // // Put together by Michael Sedlaczek, Gone Coding on 2011-02-22 //
On iOS, encryption logic is modified from the original NSData+AESCrypt is modified as:
@implementation NSData (AESCrypt)
- (NSData *)AES256EncryptWithKey:(NSString *)key
{
return [self AES256EncryptWithKey:key ECB:false];
}
- (NSData *)AES256EncryptWithKey:(NSString *)key ECB:(Boolean) ecb
{
// 'key' should be 32 bytes for AES256, will be null-padded otherwise
char keyPtr[kCCKeySizeAES256 + 1]; // room for terminator (unused)
bzero( keyPtr, sizeof( keyPtr ) ); // fill with zeroes (for padding)
// fetch key data
[key getCString:keyPtr maxLength:sizeof( keyPtr ) encoding:NSUTF8StringEncoding];
// create results buffer with extra space for padding
NSUInteger dataLength = [self length];
size_t bufferSize = dataLength + kCCKeySizeAES256;
void *buffer = malloc( bufferSize );
size_t numBytesEncrypted = 0;
NSString *hardCodeIV = @"1111111111111111";
char iv[17];
bzero( iv, sizeof( iv ) );
[hardCodeIV getCString:iv maxLength:sizeof(iv) encoding:NSUTF8StringEncoding];
//CBC
CCCryptorStatus cryptStatus = kCCSuccess;
if (ecb == false)
{
cryptStatus = CCCrypt( kCCEncrypt, kCCAlgorithmAES, kCCOptionPKCS7Padding,
keyPtr, kCCKeySizeAES256,
iv ,
[self bytes], dataLength,
buffer, bufferSize,
&numBytesEncrypted );
} else
{
// ECB
cryptStatus = CCCrypt( kCCEncrypt, kCCAlgorithmAES128, kCCOptionPKCS7Padding | kCCOptionECBMode,
keyPtr, kCCKeySizeAES256,
NULL,
[self bytes], dataLength,
buffer, bufferSize,
&numBytesEncrypted );
}
if( cryptStatus == kCCSuccess )
{
//the returned NSData takes ownership of the buffer and will free it on deallocation
return [NSData dataWithBytesNoCopy:buffer length:numBytesEncrypted];
}
free( buffer ); //free the buffer
return nil;
}
- (NSData *)AES256DecryptWithKey:(NSString *)key
{
return [self AES256DecryptWithKey:key ECB:false];
}
- (NSData *)AES256DecryptWithKey:(NSString *)key ECB:(Boolean) ecb
{
// 'key' should be 32 bytes for AES256, will be null-padded otherwise
char keyPtr[kCCKeySizeAES256+1]; // room for terminator (unused)
bzero( keyPtr, sizeof( keyPtr ) ); // fill with zeroes (for padding)
// fetch key data
[key getCString:keyPtr maxLength:sizeof( keyPtr ) encoding:NSUTF8StringEncoding];
NSUInteger dataLength = [self length];
size_t bufferSize = dataLength + kCCKeySizeAES256;
void *buffer = malloc( bufferSize );
size_t numBytesDecrypted = 0;
NSString *hardCodeIV = @"1111111111111111";
char iv[17];
bzero( iv, sizeof( iv ) );
[hardCodeIV getCString:iv maxLength:sizeof(iv) encoding:NSUTF8StringEncoding];
CCCryptorStatus cryptStatus = kCCSuccess;
if (ecb == false)
{
cryptStatus = CCCrypt( kCCDecrypt, kCCAlgorithmAES, kCCOptionPKCS7Padding,
keyPtr, kCCKeySizeAES256,
iv ,
[self bytes], dataLength,
buffer, bufferSize,
&numBytesDecrypted );
} else {
cryptStatus = CCCrypt( kCCDecrypt, kCCAlgorithmAES128, kCCOptionPKCS7Padding | kCCOptionECBMode,
keyPtr, kCCKeySizeAES256,
NULL,
[self bytes], dataLength,
buffer, bufferSize,
&numBytesDecrypted );
}
if( cryptStatus == kCCSuccess )
{
//the returned NSData takes ownership of the buffer and will free it on deallocation
return [NSData dataWithBytesNoCopy:buffer length:numBytesDecrypted];
}
free( buffer ); //free the buffer
return nil;
}
The resulting NSData element is then base64 encoded using some helper classes (used unmodified from the class) as:
static char encodingTable[64] =
{
'A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P',
'Q','R','S','T','U','V','W','X','Y','Z','a','b','c','d','e','f',
'g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v',
'w','x','y','z','0','1','2','3','4','5','6','7','8','9','+','/'
};
+ (NSData *)dataWithBase64EncodedString:(NSString *)string
{
return [[NSData allocWithZone:nil] initWithBase64EncodedString:string];
}
- (id)initWithBase64EncodedString:(NSString *)string
{
NSMutableData *mutableData = nil;
if( string )
{
unsigned long ixtext = 0;
unsigned long lentext = 0;
unsigned char ch = 0;
unsigned char inbuf[4], outbuf[3];
short i = 0, ixinbuf = 0;
BOOL flignore = NO;
BOOL flendtext = NO;
NSData *base64Data = nil;
const unsigned char *base64Bytes = nil;
// Convert the string to ASCII data.
base64Data = [string dataUsingEncoding:NSASCIIStringEncoding];
base64Bytes = [base64Data bytes];
mutableData = [NSMutableData dataWithCapacity:base64Data.length];
lentext = base64Data.length;
while( YES )
{
if( ixtext >= lentext ) break;
ch = base64Bytes[ixtext++];
flignore = NO;
if( ( ch >= 'A' ) && ( ch <= 'Z' ) ) ch = ch - 'A';
else if( ( ch >= 'a' ) && ( ch <= 'z' ) ) ch = ch - 'a' + 26;
else if( ( ch >= '0' ) && ( ch <= '9' ) ) ch = ch - '0' + 52;
else if( ch == '+' ) ch = 62;
else if( ch == '=' ) flendtext = YES;
else if( ch == '/' ) ch = 63;
else flignore = YES;
if( ! flignore )
{
short ctcharsinbuf = 3;
BOOL flbreak = NO;
if( flendtext )
{
if( ! ixinbuf ) break;
if( ( ixinbuf == 1 ) || ( ixinbuf == 2 ) ) ctcharsinbuf = 1;
else ctcharsinbuf = 2;
ixinbuf = 3;
flbreak = YES;
}
inbuf [ixinbuf++] = ch;
if( ixinbuf == 4 )
{
ixinbuf = 0;
outbuf [0] = ( inbuf[0] << 2 ) | ( ( inbuf[1] & 0x30) >> 4 );
outbuf [1] = ( ( inbuf[1] & 0x0F ) << 4 ) | ( ( inbuf[2] & 0x3C ) >> 2 );
outbuf [2] = ( ( inbuf[2] & 0x03 ) << 6 ) | ( inbuf[3] & 0x3F );
for( i = 0; i < ctcharsinbuf; i++ )
[mutableData appendBytes:&outbuf[i] length:1];
}
if( flbreak ) break;
}
}
}
self = [self initWithData:mutableData];
return self;
}
#pragma mark -
- (NSString *)base64Encoding
{
return [self base64EncodingWithLineLength:0];
}
- (NSString *)base64EncodingWithLineLength:(NSUInteger)lineLength
{
const unsigned char *bytes = [self bytes];
NSMutableString *result = [NSMutableString stringWithCapacity:self.length];
unsigned long ixtext = 0;
unsigned long lentext = self.length;
long ctremaining = 0;
unsigned char inbuf[3], outbuf[4];
unsigned short i = 0;
unsigned short charsonline = 0, ctcopy = 0;
unsigned long ix = 0;
while( YES )
{
ctremaining = lentext - ixtext;
if( ctremaining <= 0 ) break;
for( i = 0; i < 3; i++ )
{
ix = ixtext + i;
if( ix < lentext ) inbuf[i] = bytes[ix];
else inbuf [i] = 0;
}
outbuf [0] = (inbuf [0] & 0xFC) >> 2;
outbuf [1] = ((inbuf [0] & 0x03) << 4) | ((inbuf [1] & 0xF0) >> 4);
outbuf [2] = ((inbuf [1] & 0x0F) << 2) | ((inbuf [2] & 0xC0) >> 6);
outbuf [3] = inbuf [2] & 0x3F;
ctcopy = 4;
switch( ctremaining )
{
case 1:
ctcopy = 2;
break;
case 2:
ctcopy = 3;
break;
}
for( i = 0; i < ctcopy; i++ )
[result appendFormat:@"%c", encodingTable[outbuf[i]]];
for( i = ctcopy; i < 4; i++ )
[result appendString:@"="];
ixtext += 3;
charsonline += 4;
if( lineLength > 0 )
{
if( charsonline >= lineLength )
{
charsonline = 0;
[result appendString:@"n"];
}
}
}
return [NSString stringWithString:result];
}
The resulting base64 encoded string is then sent to the cloud where python pycryptodome can decrypt it as:
def aesCBCEncrypt(self,key,stri):
if isinstance(key, str):
key = key.encode('utf-8')
cipher = AES.new(key, AES.MODE_CBC, self.nonce) # , nonce=self.nonce)
if isinstance(stri, str):
data = stri.encode('utf-8')
try:
data = pad(data, 16)
ciphertext = cipher.encrypt(data)
ciphertext = b64encode(ciphertext)
ret = ciphertext.decode('utf-8')
except:
print("Some Error")
ret = ""
return ret
def aesCBCDecrypt(self,key,data):
if isinstance(key, str):
key = key.encode('utf-8')
cipher = AES.new(key, AES.MODE_CBC, self.nonce) # , nonce=self.nonce)
if isinstance(data, str):
data = data.encode('utf-8')
data = b64decode(data)
try:
result = cipher.decrypt(data)
result = unpad(result, 16)
ret = result.decode('utf-8')
except:
print("Some Error")
ret = ""
return ret
def aesECBEncrypt(self,key,stri):
if isinstance(key, str):
key = key.encode('utf-8')
cipher = AES.new(key, AES.MODE_ECB) # , nonce=self.nonce)
if isinstance(stri, str):
data = stri.encode('utf-8')
try:
data = pad(data,16)
ciphertext = cipher.encrypt(data)
ciphertext = b64encode(ciphertext)
ret = ciphertext.decode('utf-8')
except:
print("Some Error")
ret = ""
return ret
def aesECBDecrypt(self,key,data):
if isinstance(key, str):
key = key.encode('utf-8')
cipher = AES.new(key, AES.MODE_ECB) #, nonce=self.nonce)
if isinstance(data, str):
data = data.encode('utf-8')
data = b64decode(data)
try:
result = cipher.decrypt(data)
result = unpad(result,16)
ret = result.decode('utf-8')
except:
print("Some Error")
ret = ""
return ret
It is extremely important that the IV (aka nonce) is the same on iOS and Python when using CBC – it will not work otherwise. This is set in this case to a string of 16 ‘1’ characters terminated with a null. This is not really a secret key in itself, but it is likely worth changing it and securing it (possibly sending it as well with asymmetric, RSA in my case, encryption). The AES however is the critical key and should certainly be sent encrypted between the devices.
Finally, I’d recommend using the CBC even though the IV needs to be considered, as it is more secure. And when I have time I will look into integration of the Swift only Apple Crypto Kit library to support other forms as well…