Skip to content

Fix va_list on arm, and 32-bit CGFloat.NativeType #603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Fix va_list on arm, and 32-bit CGFloat.NativeType #603

wants to merge 1 commit into from

Conversation

hpux735
Copy link
Contributor

@hpux735 hpux735 commented Aug 29, 2016

This PR addresses a few problems that CF has accumulated on arm-linux platforms. One obviates the need for #399 by @karwa (and makes it hopefully more palatable). Also, a more recent addition of an import of stdio.h expresses a similar issue with the redefinition of va_list.

Another issue addressed with this PR is the fact that CGFloat.NativeType is now Float on32-bit systems and Double on 64-bit, thanks to #585 . There is a function in NSGeometry that assumed that CGFloat.native would always return Double. It's intermediate variables are now CGFloat.NativeType.

Finally, NSXML has a constant bitfield assignment that has been mentioned is redundant (see 998d5c0#commitcomment-18697699 ), but causes issues when used with signed 32-bit types. I've removed that.

@compnerd
Copy link
Member

Is there no better way to deal with the va_list conflict other than removing the stdio.h removal? It feels somewhat fragile.

@compnerd
Copy link
Member

@swift-ci please smoke test

@hpux735
Copy link
Contributor Author

hpux735 commented Aug 29, 2016

I absolutely agree. I haven't found one. I don't know why the GCC version (installed in /usr/include/stdio.h) isn't respecting the __GNUC__ and _VA_LIST_DEFINED macro definition states. I've tried explicitly undefining them (and indeed, I don't know why __GNUC__ would be defined in the first place).

@hpux735
Copy link
Contributor Author

hpux735 commented Aug 29, 2016

More context/discussion available here #399

@compnerd
Copy link
Member

compnerd commented Aug 29, 2016

The header /usr/include/stdio.h is not provided by gcc, but rather glibc. That would explain why the discrepancy exists. The version that Im looking at does respect those macros (__GNUC__ and _VA_LIST_DEFINED).

  #if defined __USE_XOPEN || defined __USE_XOPEN2K8
  #  ifdef __GNUC__
  #   ifndef _VA_LIST_DEFINED
  typedef _G_va_list va_list;
  #   define _VA_LIST_DEFINED
  #  endif
  # else
  #  include <stdarg.h>
  # endif
  #endif

__GNUC__ is a builtin define, so using that is sketchy. I think that we should be able to avoid this via _VA_LIST_DEFINED.

@hpux735
Copy link
Contributor Author

hpux735 commented Aug 29, 2016

I see what you're looking at, too. What I mean by not respecting it is that if I try to ensure that, for example, __GNUC__ is not defined, and/or _VA_LIST_DEFINED is defined prior to including stdio.h the _G_va_list version of va_list is still typedef-ed. I don't know why this is. I don't know if those macros are special (I don't see how they could be) and cannot be manipulated reliably in the scope I was working in.

@compnerd
Copy link
Member

Header search paths are incorrect is my guess. Its probably that the resource dir is being mis-ordered.

$ armv7-unknown-linux-gnueabi-gcc -P -E -x c - -o - <<< '#include <stdio.h>' | grep 'typedef.*va_list'
typedef __builtin_va_list __gnuc_va_list;
typedef __gnuc_va_list va_list;
$ armv7-unknown-linux-gnueabihf-gcc -E -x c - -o - <<< '#include <stdio.h>' | grep 'typedef.*va_list'
typedef __builtin_va_list __gnuc_va_list;
typedef __gnuc_va_list va_list;
$ armv7-unknown-linux-gnueabi-clang -E -x c - -o - <<< '#include <stdio.h>' | grep 'typedef.*va_list'
typedef __builtin_va_list va_list;
typedef __builtin_va_list __gnuc_va_list;
typedef __gnuc_va_list va_list;
$ armv7-unknown-linux-gnueabihf-clang -E -x c - -o - <<< '#include <stdio.h>' | grep 'typedef.*va_list'
typedef __builtin_va_list va_list;
typedef __builtin_va_list __gnuc_va_list;
typedef __gnuc_va_list va_list;

@hpux735
Copy link
Contributor Author

hpux735 commented Aug 29, 2016

That's a great find!

I'd love to use that as the seed for a proper fix. I don't know where the resource dir is configured, though. Do you have a hint?

@compnerd
Copy link
Member

You should be able to use cc -v. Both clang and gcc support that and should tell you the header search order. cc -v -x c -E /dev/null -o /dev/null should be sufficient. As to when it is configured, thats at compile time.

@hpux735
Copy link
Contributor Author

hpux735 commented Aug 30, 2016

What seems strange to me is that the clang 3.6 behavior on x86-64 linux:

$ clang -E -x c - -o - <<< '#include <stdio.h>' | grep 'typedef.*va_list'
typedef __builtin_va_list va_list;
typedef __builtin_va_list __gnuc_va_list;
typedef __gnuc_va_list va_list;

Is identical to the clang 3.6 behavior on armv7l linux:

$ clang -E -x c - -o - <<< '#include <stdio.h>' | grep 'typedef.*va_list'
typedef __builtin_va_list va_list;
typedef __builtin_va_list __gnuc_va_list;
typedef __gnuc_va_list va_list;

I don't follow why there isn't a problem with va_list on x86-64 linux when there is on arm.

I think I see what you're getting at with the include search path when I compare clang 3.6 to gcc, however, as the clang 3.6 search order is:

 /usr/local/include
 /usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include
 /usr/bin/../lib/gcc/arm-linux-gnueabihf/4.8/include
 /usr/include/arm-linux-gnueabihf
 /usr/include

whereas GCC is:

 /usr/lib/gcc/arm-linux-gnueabihf/4.8/include
 /usr/local/include
 /usr/lib/gcc/arm-linux-gnueabihf/4.8/include-fixed
 /usr/include/arm-linux-gnueabihf
 /usr/include

I bet had /usr/lib/llvm-3.6/bin/../lib/clang/3.6.0/include been processed prior to /usr/local/include we wouldn't be in this boat. I don't have a lot of hope for changing the include search order in the debian-armhf published clang in the near term.

Doesn't swift get compiled with its own clang 3.9 compiler?

@karwa
Copy link
Contributor

karwa commented Aug 30, 2016

@hpux735 Could that be a bug? Aren't the clang headers supposed to always come first? Trying the same on a mac (don't have an x86 linux box available, but it seems to be the same: http://stackoverflow.com/a/33485647):

/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/8.0.0/include
 /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/usr/include
 /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks (framework directory)

Otherwise, yeah, you'll get redefinition issues. Hmmm... interesting. I've seen the issue crop up occasionally over the years, but you'd think someone would have done something by now.

@hpux735
Copy link
Contributor Author

hpux735 commented Aug 31, 2016

Right, I'd think it would be, too. I have a nagging feeling it's like that for a reason (perhaps better source compatibility with GCC?), but I just don't know.

Either way, for the moment at least, we just have to deal with it. :)

@karwa
Copy link
Contributor

karwa commented Aug 31, 2016

I'm not sure the search order is the problem. My cross-compile sysroot doesn't even have a usr/local directory, and the search paths when compiling Foundation look okay:

Target: armv7-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: 
Found candidate GCC installation: /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/lib/gcc/arm-linux-gnueabihf/4.8
Selected GCC installation: /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/lib/gcc/arm-linux-gnueabihf/4.8
Candidate multilib: .;@m32
Selected multilib: .;@m32
ignoring nonexistent directory "/Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/local/include"
ignoring nonexistent directory "/Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/include"
#include "..." search starts here:
#include <...> search starts here:
 /Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift
 /Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift/clang/include
 /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include/arm-linux-gnueabihf
 /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include
End of search list.
clang -cc1 version 3.9.0 based upon LLVM 3.9.0svn default target x86_64-apple-macosx10.9
ignoring nonexistent directory "/Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/local/include"
ignoring nonexistent directory "/Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/include"
ignoring duplicate directory "/Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift"
ignoring duplicate directory "/Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include/libxml2"
ignoring duplicate directory "/Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include/curl"
#include "..." search starts here:
#include <...> search starts here:
 /Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift
 /Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift/linux
 /Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation
 /Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift
 /Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7
 /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include/libxml2
 /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include/curl
 /Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift/clang/include
 /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include/arm-linux-gnueabihf
 /Spring/Projects/3rdParty/OpenSource/Apple/cross-compile/sysroots/linux-armv7/usr/include
End of search list.

stdio.h checks _VA_LIST_DEFINED (whereas clang's stdarg defines _VA_LIST but doesn't define the _DEFINED variable, even though clang's stdalign sets similar variables). If I add it, I successfully prevent stdio redefining it, but instead see crazy errors about the exact same typedef conflicting with itself (see last 2 lines):

<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "CoreFoundation.h"
         ^
/Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CoreFoundation.h:58:10: note: in file included from /Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CoreFoundation.h:58:
#include <CoreFoundation/CFCalendar.h>
         ^
/Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CFCalendar.h:21:10: note: in file included from /Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CFCalendar.h:21:
#include <CoreFoundation/CFTimeZone.h>
         ^
/Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CFTimeZone.h:23:10: note: in file included from /Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CFTimeZone.h:23:
#include <CoreFoundation/CFString.h>
         ^
/Spring/Projects/3rdParty/OpenSource/Apple/swift-corelibs-foundation/../build/Ninja-ReleaseAssert/foundation-linux-armv7/Foundation/usr//lib/swift/CoreFoundation/CFString.h:276:123: error: reference to 'va_list' is ambiguous
CFStringRef CFStringCreateWithFormatAndArguments(CFAllocatorRef alloc, CFDictionaryRef formatOptions, CFStringRef format, va_list arguments) CF_FORMAT_FUNCTION(3,0);
                                                                                                                          ^
/Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift/clang/include/stdarg.h:30:27: note: candidate found by name lookup is 'va_list'
typedef __builtin_va_list va_list;
                          ^
/Spring/Projects/3rdParty/OpenSource/Apple/build/Ninja-ReleaseAssert/swift-linux-armv7/lib/swift/clang/include/stdarg.h:30:27: note: candidate found by name lookup is 'va_list'
typedef __builtin_va_list va_list;
                          ^

Urgh. Shot in the dark, maybe @ddunbar, as a former clangster with an interest in cross-compilation, can help?

This might be helpful, from GCC's stdarg.h:

/* We deliberately do not define va_list when called from
   stdio.h, because ANSI C says that stdio.h is not supposed to define
   va_list.  stdio.h needs to have access to that data type, 
   but must not use that name.  It should use the name __gnuc_va_list,
   which is safe because it is reserved for the implementation.  */

@Coeur
Copy link
Contributor

Coeur commented Sep 14, 2016

32-bit CGFloat.NativeType was addressed by #632, so this part of the PR can be removed.

@parkera
Copy link
Contributor

parkera commented Oct 6, 2016

So, two things now:

  1. the NSXML part has been fixed as mentioned by @Coeur.
  2. Did we come to a conclusion on if this is an issue to be fixed by changing the include order for the compiler? I'm still a bit nervous about dropping the include of stdio. It should be harmless at best.

@hpux735
Copy link
Contributor Author

hpux735 commented Oct 6, 2016

I'm not aware of a consensus on stdio.h. In libdispatch, I recall the patch having been applied. Here, however, it's still necessary. I haven't found a resolution to it via any other method to date.

@parkera
Copy link
Contributor

parkera commented Nov 29, 2016

@hpux735 what's going on with this PR?

@hpux735
Copy link
Contributor Author

hpux735 commented Nov 29, 2016

I haven't had any time to work on it. I'm still not aware of any alternate means for moving past it.

Subsequently, I've made a fork of Swift at swift-arm that has the patch. A similar PR was accepted over on the dispatch side.

It would be great if we could merge this, because this PR is the majority of changes that justify the swift-arm fork.

@threeway
Copy link

I was playing around attempting to get swift running on an ODROID-C2 which is arm64 running a Debian derivative, and I've run into this (or a similar) issue - unfortunately, the "easy" fix above, does not work with arm64 and linux.

Foundation/NSString.swift:1160:106: error: cannot convert value of type 'CVaListPointer' to expected argument type 'va_list' (aka '__va_list')
        let str = CFStringCreateWithFormatAndArguments(kCFAllocatorSystemDefault, nil, format._cfObject, argList)!
                                                                                                         ^~~~~~~
Foundation/NSString.swift:1168:148: error: cannot convert value of type 'CVaListPointer' to expected argument type 'va_list' (aka '__va_list')
                str = CFStringCreateWithFormatAndArguments(kCFAllocatorSystemDefault, unsafeBitCast(loc, to: CFDictionary.self), format._cfObject, argList)
                                                                                                                                                   ^~~~~~~
Foundation/NSString.swift:1173:106: error: cannot convert value of type 'CVaListPointer' to expected argument type 'va_list' (aka '__va_list')
            str = CFStringCreateWithFormatAndArguments(kCFAllocatorSystemDefault, nil, format._cfObject, argList)
                                                                                                         ^~~~~~~
Foundation/NSString.swift:1180:100: error: cannot convert value of type 'CVaListPointer' to expected argument type 'va_list' (aka '__va_list')
            CFStringCreateWithFormatAndArguments(kCFAllocatorSystemDefault, nil, format._cfObject, vaPtr)

Unfortunately, I don't know enough about arm64 to know why. https://bugs.swift.org/browse/SR-2239 seems to be similar, if not the same issue.

@alblue
Copy link
Contributor

alblue commented Oct 5, 2017

This pull request has gone stale, closing.

@alblue alblue closed this Oct 5, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants