Interfaces and wrappers: BLAS

[awvwgk]

Description

To support linear algebra operations we want to have interfaces/wrappers for BLAS and LAPACK, maybe even to BLACS, PBLAS and ScaLAPACK at some point. Before we start working on a high-level API, there are additional stages to consider. For this issue I want to focus on BLAS only, because the size of the problem is much more limited than the other linear algebra libraries and we can flesh out the workflow

1. How to include BLAS in stdlib?
   • building (reference) BLAS from source with fpm and/or CMake sounds like a bad idea (performance), but might be a fallback
   • linking against system BLAS and optimized BLAS libraries like MKL or OpenBLAS
2. How due we provide the BLAS functionality?
   • interface modules like BLAS95 are available
   • do we reproduce the actual reference BLAS interface (doesn't include intent in the dummy arguments)
   • do we overload the names to drop the s/d/c/z prefixes
   • are the interfaces part of stdlib or bundled from a separate project
3. How do we abstract the operations available in BLAS?
   • how many layers of abstractions do we want?
   • just simple wrapper to infer dimensions
   • overloaded operators

[arjenmarkus]

One additional thing: as the source code does not use modules, we are "polluting" the namespace with the names of the BLAS routines. So an additional step might be to put the individual routines into modules to avoid that. Another aspect is the use of routines such as DMACH1 (not in LAPACK and BLAS, but similar routines are). Some of these venerable packages require you to adjust the code for specific machines. With Fortran in its current incarnation we can rely on intrinsic functions for most of these. Op vr 2 jul. 2021 om 13:52 schreef Sebastian Ehlert < ***@***.***>:

…

*Description* To support linear algebra operations we want to have interfaces/wrappers for BLAS and LAPACK, maybe even to BLACS, PBLAS and ScaLAPACK at some point. Before we start working on a high-level API, there are additional stages to consider. For this issue I want to focus on BLAS only, because the size of the problem is much more limited than the other linear algebra libraries and we can flesh out the workflow 1. How to include BLAS in stdlib? - building (reference) BLAS from source with fpm and/or CMake sounds like a bad idea (performance), but might be a fallback - linking against system BLAS and optimized BLAS libraries like MKL or OpenBLAS 2. How due we provide the BLAS functionality? - interface modules like BLAS95 are available - do we reproduce the actual reference BLAS interface (doesn't include intent in the dummy arguments) - do we overload the names to drop the s/d/c/z prefixes - are the interfaces part of stdlib or bundled from a separate project 3. How do we abstract the operations available in BLAS? - how many layers of abstractions do we want? - just simple wrapper to infer dimensions - overloaded operators — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#450>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YR6EX4HDKZCIZD5Y7ITTVWSADANCNFSM47WSLGFA> .

[awvwgk]

@arjenmarkus I think the namespace pollution can't be helped, at least if we want to rely on optimized BLAS libraries. Putting the routines into modules would produce ABI issues unless we export them with bind(C), which kind of defeats the purpose of cleaning up the global namespace with modules in the first place.

[ivan-pi]

It will be good to study how NumPy embeds BLAS: Accelerated BLAS/LAPACK libraries

Kind of ironically for the fallback BLAS NumPy depends on a f2c'd version of BLAS which is patched to replace any new constructs with legacy constructs that work with f2c.

[awvwgk]

I was surprised to see a new release of LAPACK 3.10.0 on netlib just a few days ago, now also featuring free format source code. Therefore, I doubt that NumPy's f2c'ing strategy will be viable in the long run, maybe the transpilers planned for LFortran will be a viable alternative for NumPy at some point.

[arjenmarkus]

The fact that the code is on github as well, makes life a lot easier for fpm as well. Op vr 2 jul. 2021 om 16:17 schreef Sebastian Ehlert < ***@***.***>:

…

I was surprised to see a new release of LAPACK 3.10.0 on netlib <http://netlib.org/lapack/lapack-3.10.0.html> just a few days ago, now also featuring free format source code. Therefore, I doubt that NumPy's f2c'ing strategy will be viable in the long run, maybe the transpilers planned for LFortran will be a viable alternative for NumPy at some point. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#450 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAN6YR2O4UBJCS4EKBINZODTVXC6BANCNFSM47WSLGFA> .

[awvwgk]

Is there interest in moving this project forward? I have an implementation of a pretty complete BLAS interface library as fpm package, which could be included in stdlib: https://github.com/awvwgk/blas-interface

There are two variants in this package, a pure variant with added intent arguments or a plain variant reflecting exactly the interface exported by the reference LAPACK project.

[perazz]

Integrating BLAS and LAPACK support in stdlib

Apologies if I hijack this discussion for updating it, there are already several BLAS/LAPACK issues open.

I agree with @awvwgk on the need to link against platform-optimized libraries. To do this, I think we ought to start simple, with a build of the reference BLAS/LAPACK, because:

• focus on the high-level API now, and on the more peformant library options later
• external linking options are not currently manageable for the stdlib-fpm branch: a Fortran-based fallback implementation that fpm can build today is necessary IMHO.

I'm using numpy as an example: it has several options managed by the build process (fpm can't do this now), but anyways uses f2c copies of BLAS and LAPACK embedded in the repo in case none are available.

So I've been putting together a nearly-complete, Modern-Fortran version of BLAS and LAPACK which is ~90% automated from the Reference-LAPACK repo (license allows embedding in code provided that copyright is retained). Please note this is not an interface, but just a modern Fortran reimplementation:
https://github.com/perazz/fortran-lapack

I would love to see that merge into stdlib. (note that we may then adopt @awvwgk's interfaces IMHO).
My questions are:

• Is there any support for this idea to move forward?
• Should this Modernized BLAS/LAPACK be: 1) part of stdlib; 2) an independent Fortran-lang repo? 3) kept in sync with Netlib-lapack, or we may be begin branching our own Modern Fortran implementation from it?
• I also like the idea of an fpm metapackage that could handle different linking options, but that would come later I think.
• Modularized BLAS/LAPACK functions do not pollute the namespace, in case somewhere else the C/F77-style names are used, looks nice compared to the F77 version?

Thanks and regards.
@fortran-lang/stdlib @fortran-lang/admins

[jvdp1]

Thank you @perazz for this work!

So I've been putting together a nearly-complete, Modern-Fortran version of BLAS and LAPACK which is ~90% automated from the Reference-LAPACK repo (license allows embedding in code provided that copyright is retained). Please note this is not an interface, but just a modern Fortran reimplementation: https://github.com/perazz/fortran-lapack

Impressive work!

I would love to see that merge into stdlib. (note that we may then adopt @awvwgk's interfaces IMHO). My questions are:

• Is there any support for this idea to move forward?

I support totally this, and would be a very good addition for stdlib and hopefully the Fortran community.

• Should this Modernized BLAS/LAPACK be: 1) part of stdlib; 2) an independent Fortran-lang repo?

I think it would be good to integrate it in stdlib. However, during its development, a separate branch might be needed.

3. kept in sync with Netlib-lapack, or we may be begin branching our own Modern Fortran implementation from it?

• I also like the idea of an fpm metapackage that could handle different linking options, but that would come later I think.

Once the issues regarding fpm and stdlib due to the use of fypp are solved, such linking options will be possible with fpm, right? I guess it should be already possible with CMake.

• Modularized BLAS/LAPACK functions do not pollute the namespace, in case somewhere else the C/F77-style names are used, looks nice compared to the F77 version?

Not sure to understand your question.

[perazz]

Thank you @jvdp1, let's make it happen then!

I see that I can make progress pretty fast on the separate repo, so take your time, and when ready, we can merge it to the linear algebra branch with a PR. I think it may even be more convenient to leave it as a separate repo until folks want try out and give feedback the modern APIs, because they can do it easily with fpm. Both options will work.

Once the issues regarding fpm and stdlib due to the use of fypp are solved, such linking options will be possible with fpm, right? I guess it should be already possible with CMake.

I think so. IMHO All that's needed is the ability to define/undefine a macro flag to tell the library whether the internal implementation, or the external library, should be used. This is no problem for CMake, but maybe not even for the stdlib-fpm branch: one could just require the user to add a well-named macro at the package level, I imagine something like

[dependencies]
stdlib = "*"

[preprocess]
[preprocess.cpp]
macros = ["STDLIB_WITH_EXTERNAL_BLAS"]

[perazz]

Not sure to understand your question.

I meant that if one wants to use the local implementation, they can just call use stdlib_linalg_blas, only: stdlib_dgeev from the internal implementation because there will never be duplicate routines with the default dgeev name.