warm start performance

[tomerd]

this issue is to track details re. warm start performance

[tomerd]

current benchmark based on ./scripts/performance_test.sh

-----------------------------
results
-----------------------------
string, cold: 32658752 (ns)
string, warm: 489904 (ns)
json, cold: 34155567 (ns)
json, warm: 626493 (ns)
-----------------------------

[drexin]

So I managed to statically link the project and got following numbers:

-----------------------------
results
-----------------------------
string, cold: 8676418 (ns)
string, warm: 222640 (ns)
json, cold: 10234056 (ns)
json, warm: 399833 (ns)
-----------------------------

I think that looks very competitive ;)

[fabianfett]

What?! 😱 @drexin this is super awesome!

[drexin]

So to link statically, I first switched to Swift 5.2-dev, because previous versions did not come with all the necessary static libraries. Then we need to install some additional dependencies (these are the Ubuntu package names):

• libicu-dev
• uuid-dev

And then we need to modify the file /usr/lib/swift_static/linux/static-executable-args.lnk to have the following content:

/usr/lib/swift_static/linux/static-executable-args.lnk
-static
-lswiftCore
-lswiftImageInspectionShared
-Xlinker
--defsym=__import_pthread_self=pthread_self
-Xlinker
--defsym=__import_pthread_once=pthread_once
-Xlinker
--defsym=__import_pthread_key_create=pthread_key_create
-lpthread
-latomic
-lDispatchStubs
-lCoreFoundation
-lFoundation
-luuid
-licui18nswift
-licuucswift
-licudataswift
-licui18n
-licuuc
-licudata
-ldl
-lstdc++
-lm

After all that we should be able to compile with swift build -Xswiftc -static-executable -c release.

I'm hoping to get these changes upstream soon. Obviously we can't just add all the libraries unconditionally, because it depends on the libraries we actually use. I'll see how the driver decides what's actually necessary.

[fabianfett]

@drexin With this list it looks to me that I don't even need to install swift on my target Amazon Linux 2. Is this correct?

[drexin]

That is correct!

[tomerd]

@fabianfett until we have official AL2 images, would be cool if you could create a 5.2-dev version with this patch, so we can try to build & test the test payload on real lambda. possible?

[tomerd]

@drexin these are amazing results, can't wait to use it

[fabianfett]

@tomerd my macbook is really looking forward to run with fans spinning for at least 3 hours today. 😉

Will definitely try this out and upload the binaries at my site.

@drexin I will use just plain master for this. If there is a better/more stable branch I should checkout, give me a hint.

[drexin]

master is fine, or 5.2

[tomerd]

@tomerd my macbook is really looking forward to run with fans spinning for at least 3 hours today.

use aws?!

[fabianfett]

@drexin I tried to compile and test the swift-5.2-branch with docker (with skip-foundation-test):

https://github.com/fabianfett/amazonlinux-swift/blob/run_tests/compile/Dockerfile

Sadly I ran into an issue:

CMakeError.log
CMakeOutput.log
stdout.log

Do you know anything about this? Am I doing something wrong?

[compnerd]

@fabianfett - I think that you missed --cmark in your invocation (or your cmark checkout is out of date)

[fabianfett]

@compnerd Sorry for coming back to you but I'm not quite sure if I understand that correctly. I just reran my build with the same result.

The checkout ended with

Updating '/home/ec2-user/swift'
Tag 'swift-5.2-branch' does not exist for 'swift', just updating regularly
/home/ec2-user/swift

/home/ec2-user/swift
Current branch swift-5.2-branch is up to date.

/home/ec2-user/swift

update-checkout succeeded
cmake                              : f43a7d76c737c5bb9b903a2b1be5186c081ec21e
cmark                              : 1168665f6b36be747ffe6b7b90bc54cfc17f42b7
icu                                : fd123bf023882f07bfacf51c39111be2f946d8f8
indexstore-db                      : 14e6e25a53ca08946ae0927b6cc2b2bf36c311b9
llbuild                            : 15cb77c8b3e5ac21cbe47fd6f1f2c891d4b56093
llvm-project                       : 16b576d445bd278dace043e1f0a6e9b8eda9ddb5
ninja                              : ed7f67040b370189d989adbd60ff8ea29957231f
sourcekit-lsp                      : 7d18d3bb3841594f443868bc172bcd8a84d756b9
swift                              : 33150e32b4f5c5f9ecae234b3c3d8251877de402
swift-corelibs-foundation          : 69232d2ef971acf99938569fdd2caa8a16cb7718
swift-corelibs-libdispatch         : c992dacf3ca114806e6ac9ffc9113b19255be9fe
swift-corelibs-xctest              : 8b0eefa96c02a4cb4d3eb74e6c289eba34a744fa
swift-format                       : 5e7ef93b6b93381c6805fea57b3ccfb05f718846
swift-integration-tests            : 11f0f6e8b34ba9782b5841dbeaa207d0b4620152
swift-stress-tester                : 225a973f42140890bb49648ca090d1a53f32ff8e
swift-syntax                       : 7251727261c6d2fb225245dc20ffded8f8d0f000
swift-tools-support-core           : d08234c4cf8046786a3552143532222a56273720
swift-xcode-playground-support     : 88043d7d320f92598efb39408c3f4b1903a4fff6
swiftpm                            : 75916ca4f00c6861d8dc1b03cb5684d6fc4f8f8d

which let's me assume that cmark is checked out by the update-checkout automatically. Adding --cmark to the build-script invocation resulted in

utils/build-script --preset=buildbot_linux installable_package=/home/ec2-user/swift-package.tar.gz install_destdir=/tmp/swift

usage: build-script [-h] [-n] [--preset-file PATH] [--preset NAME]
                    [--show-presets [BOOL]] [--distcc [BOOL]]
                    [--cmake-c-launcher PATH] [--cmake-cxx-launcher PATH]
                    [-j BUILD_JOBS] [--expand-invocation [BOOL]]
                    [--swiftsyntax-install-prefix SWIFTSYNTAX_INSTALL_PREFIX]
                    [--build-dir BUILD_DIR] [--dump-config] [--reconfigure]
                    [SUBSTITUTION [SUBSTITUTION ...]]
build-script: error: unrecognized arguments: --cmark

I'm sure I made a stupid mistake here but could you please point me in the correct direction.