[SR-2879] Creating 4 calendars results in an assertion failure

[alblue]

Previous ID	SR-2879
Radar	None
Original Reporter	@alblue
Type	Bug
Status	Resolved
Resolution	Done

Additional Detail from JIRA

Votes	0
Component/s	Foundation
Labels	Bug
Assignee	@alblue
Priority	Medium

md5: 6c8273abf9d22068045fc840a683dc37

relates to:

• SR-2462 Compiler is sometimes overreleasing CF objects

Issue Description:

This code fails on Linux when assertions are enabled in a build:

```
$ swift-dev/usr/bin/swift
Welcome to Swift version 3.0-dev (LLVM b9bd56d1b8, Clang de97de0726, Swift 44b8a0bd12). Type :help for assistance.
1> import Foundation
2. _ = Calendar(identifier:.buddhist)
3. _ = Calendar(identifier:.chinese)
4. _ = Calendar(identifier:.islamic)
5. _ = Calendar(identifier:.gregorian)
repl_swift: /home/ablewitt/swift-build/swift/include/swift/Runtime/../../../stdlib/public/SwiftShims/RefCount.h:252: bool StrongRefCount::doDecrementShouldDeallocate() [ClearPinnedFlag = false]: Assertion `newval + quantum >= RC_ONE && "releasing reference with a refcount of zero"' failed.
Execution interrupted. Enter code to recover and continue.
Enter LLDB commands to investigate (type :help for assistance.)
6> :bt

• thread Fixing whitespace formatting #1: tid = 4412, 0x00007ffff6d88418 libc.so.6`__GI_raise(sig=6) + 56 at raise.c:54, name = 'repl_swift', stop reason = signal SIGABRT

• frame #0: 0x00007ffff6d88418 libc.so.6`__GI_raise(sig=6) + 56 at raise.c:54
  frame Fixing whitespace formatting #1: 0x00007ffff6d8a01a libc.so.6`__GI_abort + 362 at abort.c:89
  frame Fixing grammar issue #2: 0x00007ffff6d80bd7 libc.so.6`__assert_fail_base(fmt="", assertion=<unavailable>, file=<unavailable>, line=<unavailable>, function=<unavailable>) + 279 at assert.c:92
  frame Syntax highlighting in README.md #3: 0x00007ffff6d80c82 libc.so.6`GI_assert_fail(assertion=<unavailable>, file=<unavailable>, line=<unavailable>, function=<unavailable>) + 66 at assert.c:101
  frame Add @noescape to Synchronous Enumeration, Predicate, and Comparator Blocks #4: 0x00007ffff7ceb790 libswiftCore.so`swift_release + 80
  frame Verify contents of the file attributes dictionary #5: 0x00007ffff3d9242e libFoundation.so`__CFCalendarDeallocate + 46
  frame Fix URL in README.md #6: 0x00007ffff3ef790e libFoundation.so`Foundation.NSCalendar.__deallocating_deinit + 14
  frame Use Correct Source Index in NSArray.getObjects #7: 0x00007ffff4259372 libFoundation.so`Foundation.MutableHandle._deallocating_deinit + 18
  frame [WIP] Implement NSRangeFromString #8: 0x00007ffff7f6a098 $__lldb_expr2`main + 136
  frame Use Array.reserveCapacity in NSArray/NSMutableArray #9: 0x00000000004009c0 repl_swift`swift_once + 16
  frame Fix download link #10: 0x00007fffffffecc0
  frame Fix a bug in CFReadStreamGetBuffer for data streams #11: 0x00007ffff6d73830 libc.so.6`__libc_start_main(main=(repl_swift`main), argc=1, argv=0x00007fffffffee38, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffee28) + 240 at libc-start.c:291
  frame Several NSString constructors implemented #12: 0x00000000004009e9 repl_swift`_start + 41
  6>
  ```

If assertions are disabled, then the code appears to work (but we may have a use-after-free). Running the same code after the failure occurs results in success. It doesn't appear to matter which calendar identifiers are used (provided that they are different to each other)

[alblue]

Looking through a debugger, the error seems to happen when deallocating kFCEmptyString:

```
(lldb) disas -n __CFCalendarDeallocate
libFoundation.so`__CFCalendarDeallocate:
0x7ffff3d92400 <+0>: pushq %rbx
0x7ffff3d92401 <+1>: movq %rdi, %rbx
0x7ffff3d92404 <+4>: movq 0x18(%rbx), %rdi
0x7ffff3d92408 <+8>: testq %rdi, %rdi
0x7ffff3d9240b <+11>: je 0x7ffff3d92412 ; <+18>
0x7ffff3d9240d <+13>: callq 0x7ffff3d6f8d0 ; CFRelease
0x7ffff3d92412 <+18>: movq 0x20(%rbx), %rdi
0x7ffff3d92416 <+22>: testq %rdi, %rdi
0x7ffff3d92419 <+25>: je 0x7ffff3d92420 ; <+32>
0x7ffff3d9241b <+27>: callq 0x7ffff3d6f8d0 ; CFRelease
0x7ffff3d92420 <+32>: movq 0x28(%rbx), %rdi
0x7ffff3d92424 <+36>: testq %rdi, %rdi
0x7ffff3d92427 <+39>: je 0x7ffff3d9242e ; <+46>
0x7ffff3d92429 <+41>: callq 0x7ffff3d6f8d0 ; CFRelease
0x7ffff3d9242e <+46>: movq 0x30(%rbx), %rdi
0x7ffff3d92432 <+50>: testq %rdi, %rdi
0x7ffff3d92435 <+53>: je 0x7ffff3d9243c ; <+60>
0x7ffff3d92437 <+55>: callq 0x7ffff3d6f8d0 ; CFRelease
0x7ffff3d9243c <+60>: movq 0x38(%rbx), %rdi
0x7ffff3d92440 <+64>: testq %rdi, %rdi
0x7ffff3d92443 <+67>: je 0x7ffff3d9244b ; <+75>
0x7ffff3d92445 <+69>: popq %rbx
0x7ffff3d92446 <+70>: jmp 0x7ffff3d68c10 ; symbol stub for: ucal_close_55
0x7ffff3d9244b <+75>: popq %rbx
0x7ffff3d9244c <+76>: retq
```

This is where we are when we're just about to throw an assertion failure:

```
libswiftCore.so`swift_release:
0x7ffff7ceb740 <+0>: testq %rdi, %rdi
0x7ffff7ceb743 <+3>: je 0x7ffff7ceb76f ; <+47>
0x7ffff7ceb745 <+5>: movl $0xfffffffc, %eax ; imm = 0xFFFFFFFC
0x7ffff7ceb74a <+10>: lock
0x7ffff7ceb74b <+11>: xaddl %eax, 0x8(%rdi)
0x7ffff7ceb74f <+15>: cmpl $0x3, %eax
0x7ffff7ceb752 <+18>: jbe 0x7ffff7ceb770 ; <+48>
0x7ffff7ceb754 <+20>: cmpl $0x4, %eax
0x7ffff7ceb757 <+23>: jne 0x7ffff7ceb76f ; <+47>
0x7ffff7ceb759 <+25>: leaq 0x8(%rdi), %rcx
0x7ffff7ceb75d <+29>: movl $0x2, %edx
0x7ffff7ceb762 <+34>: xorl %eax, %eax
0x7ffff7ceb764 <+36>: lock
0x7ffff7ceb765 <+37>: cmpxchgl %edx, (%rcx)
0x7ffff7ceb768 <+40>: jne 0x7ffff7ceb76f ; <+47>
0x7ffff7ceb76a <+42>: jmp 0x7ffff7ceb6d0 ; _swift_release_dealloc
0x7ffff7ceb76f <+47>: retq
-> 0x7ffff7ceb770 <+48>: pushq %rax
0x7ffff7ceb771 <+49>: leaq 0x25219(%rip), %rdi
0x7ffff7ceb778 <+56>: leaq 0x2525e(%rip), %rsi
0x7ffff7ceb77f <+63>: leaq 0x25311(%rip), %rcx
0x7ffff7ceb786 <+70>: movl $0xfc, %edx
0x7ffff7ceb78b <+75>: callq 0x7ffff7ab3340 ; symbol stub for: __assert_fail
```

At this point %rdi claims to point to kCFEmptyString:

```
(lldb) reg r
General Purpose Registers:
rax = 0x0000000000000001
rbx = 0x0000000000448df0
rcx = 0x00007ffff46741d0 libFoundation.so`__CFRuntimeClassTable
rdx = 0x0000000000000002
rdi = 0x00007ffff466d880 libFoundation.so`__kCFEmptyString
```

It looks like it's coming from the free of the localeID:

```
static void __CFCalendarDeallocate(CFTypeRef cf) {
CFCalendarRef calendar = (CFCalendarRef)cf;
if (calendar->_identifier) CFRelease(calendar->_identifier);
if (calendar->_locale) CFRelease(calendar->_locale);
if (calendar->_localeID) CFRelease(calendar->_localeID);
if (calendar->_tz) CFRelease(calendar->_tz);
if (calendar->_cal) ucal_close(calendar->_cal);
}
```

[alblue]

It looks like the kCFEmptyString's retain count is steadily decreasing.

: p (int)swift_retainCount(&__kCFEmptyString)

{{
$ swift-dev/usr/bin/swift
Welcome to Swift version 3.0-dev (LLVM b9bd56d1b8, Clang de97de0726, Swift 44b8a0bd12). Type :help for assistance.
1> import Foundation
2> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $11 = 1
2> _ = Calendar(identifier:.gregorian)
3> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $12 = 2
3> _ = Calendar(identifier:.chinese)
4> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $13 = 1
4> _ = Calendar(identifier:.indian)
5> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $14 = 0
5> _ = Calendar(identifier:.iso8601)
repl_swift: /home/ablewitt/swift-build/swift/include/swift/Runtime/../../../stdlib/public/SwiftShims/RefCount.h:252: bool StrongRefCount::doDecrementShouldDeallocate() [ClearPinnedFlag = false]: Assertion `newval + quantum >= RC_ONE && "releasing reference with a refcount of zero"' failed.
Execution interrupted. Enter code to recover and continue.
Enter LLDB commands to investigate (type :help for assistance.)
6>
}}

[swift-ci]

Comment by W S (JIRA)

SR-2462?

[alblue]

Not sure that it's definitely related. The Calendar appears to be using kCFEmptyString for its default locale, and it doesn't look like the CF constant strings use an infinite retain count to prevent them from being removed:

https://github.com/alblue/swift-corelibs-foundation/blob/master/CoreFoundation/Base.subproj/CFInternal.h#L384-L386

I believe the error is that 'swift_release' on CF constants causes them to be removed:

1> import Foundation
2> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $11 = 1
2> : p (void)CFRelease(&__kCFEmptyString)
2> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $12 = 0
2> : p (void)CFRelease(&__kCFEmptyString)
repl_swift: /home/ablewitt/swift-build/swift/include/swift/Runtime/../../../stdlib/public/SwiftShims/RefCount.h:252: bool StrongRefCount::doDecrementShouldDeallocate() [ClearPinnedFlag = false]: Assertion `newval + quantum >= RC_ONE && "releasing reference with a refcount of zero"' failed.
error: Execution was interrupted, reason: signal SIGABRT.
The process has been returned to the state before expression evaluation.

The Swift compiler doesn't get a look in at this point, since the Calendar is doing its own retain/release under the covers. However, the original implementation was written such that it expected that the constants to never be able to be released ...

[alblue]

OK, I can fix it with an inserted CFRetain for the locale:

1> import Foundation
2> : p swift_retainCount(&__kCFEmptyString)
error: warning: got name from symbols: __kCFEmptyString
error: 'swift_retainCount' has unknown return type; cast the call to its declared return type
2> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $11 = 1
2> _ = Calendar(identifier:.gregorian)
3> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $12 = 3
3> _ = Calendar(identifier:.chinese)
4> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $13 = 3
4> _ = Calendar(identifier:.hebrew)
5> : p (int)swift_retainCount(&__kCFEmptyString)
(int) $14 = 3

[alblue]

Pushed #675 to fix