Commit Graph

4672 Commits

Author SHA1 Message Date
Vassil Vassilev
2755831fc0 Do not register optional modulemaps.
For some common libraries we provide modulemaps such as tinyxml2 and boost.
Independently whether implicit or fno-implicit-module-maps is selected we insert
an overlay and register the modulemap directly.

This patch disables this when fno-implicit-module-maps is chosen. This should
resolve ROOT-10803 introduced by root-project/root#5709
2020-06-07 10:29:08 +02:00
Axel Naumann
05bd612886 Flag an error-Transaction as kRolledBack (ROOT-10798):
Even for the ErrorOut case the Transaction should be flagged as beyond
kCompleted, to prevent it from being asserted on as "we have transaction
without errors that was not committed" in ~IncrementalParser().
2020-05-29 15:44:09 +02:00
Vassil Vassilev
34103455b9 Adjust the location of the boost modulemap. 2020-05-27 08:44:04 +02:00
Axel Naumann
02737be6d5 Enable colored diagnostics also for Ninja, also for GCC:
Ninja buffers compiler output, and compilers then think they should
not use colored output (because no terminal). Force it on them.
The same was already implemented for clang, a few lines above.

And let cling survive: it's parsing compiler output, and the ANSI
color escapes confuse the regexes.
2020-05-25 22:29:09 +02:00
Vassil Vassilev
9278757ac3 Do not try to compare two realpaths.
This conservative assert intended to capture future uses of the
shouldPermanentlyIgnore routine and check if we have specified an absolute
non-symlinked path.

Turns out that when we scan for libraries we iterate a given folder which may
move files around. Then the iterable (official) filename's real path might
differ at the point of iteration and the point of the check. Some file systems
use a temporary filename while copying/moving file and the rename it to the
real filename preserving atomicity of the operations.

It looks like this exactly happens in the description of ROOT-10791 -- the
iterator gives the expected real path of the filename which is not yet available
when we ask for the real path of that path we get something different (using the
same inode) eg: `mathsymb.ps` vs `mathsymb.ps_tmp_4810`.

Instead of asserting, let the file system handles this instead.

Kudos to Philippe Canal for investigating this sporadic failure.
2020-05-25 18:29:07 +02:00
Vassil Vassilev
dea63b95b3 Allow a Dyld per DynamicLibraryManager.
This addresses the rest of the comments of root-project/root#4717

Patch by Alexander Penev and me.
2020-05-25 06:14:06 +02:00
Vaibhav Garg
807dafd1cb Fix Thread.pcm on Windows 2020-05-25 06:14:06 +02:00
Vaibhav
ad695461b9 Fix Core.pcm on Windows 2020-05-21 19:29:06 +02:00
Simeon Ehrig
c2cd8d4dcd Backport bugfix in the ptx generator
- now the ptx code should be aligned correctly
2020-05-20 15:29:05 +02:00
Axel Naumann
6e5b2b6507 Also capture SFINAE error counts (ROOT-10754, ROOT-10777):
When doing lookup on templates, instantiation can fail.
This can be triggered during template instantiation somewhere
in clang, autoloading, cling-lookup - and SFINAE errors that
occur in cling-lookup must not bubble up to clang, or else
clang will think that there was a problem (where there was
none - just e.g. ROOT trying to autoload a bogus template).

In this concrete case, a template specialized with a lambda
was not found by clang, was tried to be autoloaded, TMetaUtils
produced a broken normalized type name, lookup on the broken
type name failed with a SFINAE error - and that ended up being
swallowed by a clang SFINAETrap by the topmost lookup.

Instead, keep SFINAE errors to ourselves.

This is an improved version of 40673e583fceda8b362a0902c6aba371ddd0cfd8
which lowers the SFINAE-reset to Lookup, as AutoParse can still
see partial template specialization scopes. It fixes ROOT-10777.
2020-05-20 12:59:06 +02:00
Axel Naumann
12259fc4e3 Make use of shared_ptr operators instead of get() (NFC). 2020-05-20 12:59:06 +02:00
Vaibhav
2fe46da7eb Make xtimec.h and xthreads.h textual in std_msvc.modulemap 2020-05-16 12:59:06 +02:00
Vaibhav Garg
8b4e4619a4 Fix std and libc modules on Windows (#5610)
[wincxxmodules] Fix std and libc modules on Windows
2020-05-14 16:29:08 +02:00
Pratyush Das
f8ccd27e67 Fix tar_required logic
If all requirements were satisfied, tar_required would always be false.
This commit fixes it.
2020-05-13 09:29:07 +02:00
Pratyush Das
87011bd783 Fix cmake requirement 2020-05-13 09:29:07 +02:00
Pratyush Das
98a1c4acad Fix svn installation 2020-05-13 09:29:07 +02:00
Pratyush Das
52f75d8768 Fix distro requirement (and installation of pip)
There might be an issue when using conda as default Python for Python3
because conda calls it pip but otherwise it is pip3.
2020-05-13 09:29:07 +02:00
Vaibhav
312b20edbf Create vcruntime.modulemap for use in Windows 2020-05-13 07:29:08 +02:00
Vassil Vassilev
e505c03038 Even when library has no symbols to offer initialize the filter.
This is part of root-project/root#4717
2020-05-07 15:59:07 +02:00
Vassil Vassilev
d7963381fe Adjust asserts.
This is part of root-project/root#4717
2020-05-07 13:59:05 +02:00
Martin Ritter
3c4c24d588 Re-enable switch to C-locale when searching for C++ includes (ROOT-10751)
Commit df0b689 simplified the search for the C++ include directories and
tried to use a locale independent regex.

However this regex doesn't work for all locales, for example in German
the string in the output is not

```
#include <...> search starts here:
```

but the word order is switched and we have

```
Suche für »#include <...>« beginnt hier:
```

As such I propose to go back to C-locale for this query to be completely
safe against weird locale settings.
2020-05-07 09:14:04 +02:00
Vassil Vassilev
e00a84804b Check if we have a bloom filter.
This fixes an assert in isBloomFilterEmpty which assumes
the bloom filter is already built.
2020-05-07 09:14:04 +02:00
Alexander Penev
bdd03109ba Improve symbol resolution.
This patch consolidates the symbol resolution facilities throughout TCling into
a new singleton class Dyld part of the cling's DynamicLibraryManager.

The new dyld is responsible for:
  * Symlink resolution -- it implements a memory efficient representation of
    the full path to shared objects allowing search at constant time O(1). This
    also fixes issues when resolving symbols from OSX where the system libraries
    contain multiple levels of symlinks.
  * Bloom filter optimization -- it uses a stohastic data structure which gives
    a definitive answer if a symbol is not in the set. The implementation checks
    the .gnu.hash section in ELF which is the GNU implementation of a bloom
    filter and uses it. If the symbol is not in the bloom filter, the
    implementation builds its own and uses it. The measured performance of the
    bloom filter is 30% speed up for 2mb more memory. The custom bloom filter on
    top of the .gnu.hash filter gives 1-2% better performance.
    The advantage for the custom bloom filter is that it works on all
    implementations which do not support .gnu.hash (windows and osx). It is also
    customizable if we want to further reduce the false positive rates
    (currently at p=2%).
  * Hash table optimization -- we build a hash table which contains all symbols
    for a given library. This allows us to avoid the fallback symbol iteration
    if multiple symbols from the same library are requested. The hash table
    optimization targets to optimize the case where the bloom filter tells us
    the symbol is *maybe* in the library.

Patch by Alexander Penev and me!
2020-05-06 22:14:06 +02:00
Vassil Vassilev
7143fce725 Support libcxx with glibc on unix.
This patch works around compatibility issues between libcxx and glibc. It should
fix the set up of the FNAL ART framework.

Thanks to Chris Green this patch should resolve ROOT-10677.
2020-04-22 06:29:07 +02:00
Vassil Vassilev
0796990e74 Enable proper installation of modulemaps; build the libc module. 2020-04-17 11:59:04 +02:00
Vassil Vassilev
a8e33d8e95 Don't mount a custom modulemap file if a module.modulemap exists
OSX has modularized sdk (libc++ included) and there we prefer the modulemaps
that libraries ship with. However, libc++ can be also used on other Unix
platforms and we should not try to mount the std.modulemap file for it.

This patch fixes ROOT-10677.
2020-04-17 08:14:07 +02:00
Axel Naumann
41dbc3be44 Add test for ROOT-10689. 2020-04-16 18:29:08 +02:00
Axel Naumann
3882f1346b Shadow only prompt definitions:
This prevents an assert (isClingShadowNamespace(D->getDeclContext()) && "D not in a __cling_N5xxx namespace?"), function hideDecl, file /build/jenkins/night/LABEL/mac1014/SPEC/soversion/V/master/root/interpreter/cling/lib/Interpreter/DefinitionShadower.cpp, line 61
2020-04-16 18:29:08 +02:00
Vassil Vassilev
ae36b2aed0 Re-attach the standard modulemaps on osx.
This should fix the regressed runtime_cxxmodules on osx.

Patch by Alexander Penev!
2020-04-06 14:14:07 +02:00
Vaibhav
9d0f9fe227 Add remaining headers compatible to MSVC 2020-04-06 13:14:04 +02:00
Vassil Vassilev
3a56ef4382 Add stub modulemaps for windows. 2020-04-06 13:14:04 +02:00
Vassil Vassilev
a7d40e8237 Bump clad version to v0.6.
The new release includes some improvements in Reverse mode:
  * Reduce the quadratic cloning complexity to linear.
  * Support variable reassignments pontentially depending on control flow.
  * Support operators `+=`, `-=`, `*=`, `/=`, `,`, `++`, `--`.
  * Allow assignments to array subscripts.
  * Support nested assignments in expressions `a = b * ((c ? d : e) = f = g);`
  * Enable differentiation of for-loops

See more at: https://github.com/vgvassilev/clad/blob/v0.6/docs/ReleaseNotes.md

This patch enables us to upgrade to llvm9. Clad supports from clang5 to clang9.
2020-04-03 11:29:05 +02:00
Vassil Vassilev
4261ea5529 Do not add overlay files when system modulemaps exist.
This patch also instruct the build system to not copy {libc,std}.modulemap in
OSX because the standard libraries are already modularized.

Fixes a regression introduced by root-project/root@603a1c3e96
2020-03-30 12:29:07 +02:00
Vassil Vassilev
632af4f5d7 Use real modulemap names if implicit module map discovery is off.
The current logic of pseudo-automatic modulemap discovery is a little fragile
as it mounts a virtual file named module.modulemap in the location where the
a give system library header is found (cuda.h, etc). However, if the libraries
are squashed into a single folder then we end up mounting a file over another
virtual file. This should be a hard error, however, on systems such as OSX some
libraries already come with modulemap files and we should just not mount our
predefined ones. This makes it very difficult to distinguish when to issue the
hard error.

While this patch is not a solution for the general when the modulemaps are
automatically discovered (-fimplicit-module-maps), we can use the real file
stems in cases where we are called with -fno-implicit-module-maps.

As a side effect this fixes the *non-recommended* squashing of library header
files into a single folder.
2020-03-20 15:44:05 +01:00
Vassil Vassilev
4d0c4f41bc Move the diagnostic client init before potential source of diagnostics.
Handling modulemap files may issue parsing errors or not found errors and our
diagnostics client should be prepared for it.

This patch fixes an assertion in clang when we see duplicate modulemap files.
2020-03-19 15:29:08 +01:00
Vassil Vassilev
37ee0fa612 Simplify and format better. NFC. 2020-02-25 21:47:06 +01:00
Vassil Vassilev
cd484295c1 Teach cling to give unique source locations.
In number of cases over the last years we have seen a need to call clang APIs
with valid source locations. The interpreter generates code and valid locations
can be problematic.

This patch allows cling to return a valid source location even when no code was
processed. This way we can provide our modules infrastructure proper source
locations to activate the module visibility rules on locations which have common
predecessor. The common predecessor is essential when we compare entities for
diagnostics or reasoning about module visibility.
2020-02-25 21:47:05 +01:00
Vassil Vassilev
c7eb512358 Add a callback for start/finish code generation.
When we are generating code, CodeGen automatically tries to complete decl's
redeclaration chain. This ends up a call to the external sources, one of which
is our global module index (GMI).

The semantics of the GMI is to aid the frontend, that is to automatically
import missing modules based on the typed *by the user* identifiers. It does
not intend to aid the code generation by any means. Currently this happens
when completing chains for existing identifiers such as 'volume' which happens
to be an indentifier part of TMVA.pcm, too. Thus, CodeGen unintentionally loads
the module of TMVA. This is an overkill, but worse -- it brings recursiveness
to the identifier resolution system.

This patch disables looking in the GMI at codegen time.
2020-02-24 11:29:13 +01:00
Vassil Vassilev
35641c2c8b Do not cache the file lookup failure.
In cases where we have the module cache path and prebuilt module path pointing
to the same location, the FileManager should not cache the module file lookup
failure because it may currently be building the module.

This patch is necessary because the global module index is built from the
module cache path and it is loaded from the prebuilt module path.

In a full explicit or implicit module build infrastructure this is not a problem.
However, in a mixed scenario (where modules for third-party dependencies are
built implicitly) such as ours this is problematic. One of the reasons is that
we cannot configure the prebuilt modules paths or module cache paths externally.
This is because the interpreter (at construction time) #includes RuntimeUniverse
which may trigger module build.

This patch allows us to refactor some of the code working around this issue.
2020-02-24 11:29:13 +01:00
Vassil Vassilev
051044c062 Implement global module indexing to improve performance.
The global module index represents an efficient on-disk hash table which stores
identifier->module mapping. Every time clang finds a unknown identifier we
are informed and we can load the corresponding module on demand.

This way we can provide minimal set of loaded modules. Currently, we see that
for hsimple.C only the half of the modules are loaded. This can be further
improved because we currently load all modules which have an identifier, that is
when looking for (for example TPad) we will load all modules which have the
identifier TPad, including modules which contain only a forward declaration of
it.

Kudos Arpitha Raghunandan (arpi-r)!
2020-02-24 11:29:13 +01:00
Axel Naumann
9220b498f3 Capture/reset expression eval context (ROOT-10511):
Without resetting the eval context to the Parser default (PotentiallyEvaluated),
recursive parsing can potentially assume wrong context and not emit referenced entities.
2020-02-19 11:30:36 +01:00
Philippe Canal
7a5894d1b8 meta/cling: consistent casing for AutoLoading 2020-02-15 17:14:14 +01:00
Oksana Shadura
e7e51e125c Adding boost and tinyxml2 modulemaps needed for CMSSW
Modulemaps are generated by https://github.com/Teemperor/boost-compile Raphael Isemann (boost modulemap is specifically tailored for CMSSW needs)
2020-02-14 00:09:11 +01:00
Sergey Linev
269608a831 Fix memory leak in cling/lib/Utils/PlatformWin.cpp
Seems to be, buffer is not released correctly.
I guess, error should be fixed in cling repository first
2020-01-30 16:29:05 +01:00
Philippe Canal
6e2db073b7 Compile Interpreter.cpp with exception on. Fix script with exception with ROOT Mutex on.
Compiling Interpreter.cpp allows RunFunction and friends to be actually seen during stack unwind after an
exception has been thrown, directly or indirectly, by interpreter code.  This allows for the RAII objects to be
properly tear down.

In particular, without this patch, EnterUserCodeRAII was not tear down and thus the callbacks were not executed.
Consequently the "Restore the ROOT global Mutex" callback was not executed leaving the Mutex in an invalid state.

In case of ART application, in most cases, they customize the ROOT error handler to throw an exception.  This
resulted (without this fix) in crash when import a GDML file with an error in it.

In practice what we have is:

    call to TGeo Import
       which calls the interpreter for some of its functionality
          which calls gdml code
            which reports an error
               which leads the error handler to thrown an exception.
       ... some of the stack are properly unwound ... some are not (because they were not compiled with exception support on)  ....
       ... so the ROOT Mutex goes into an incorrect state ...
       ... unwinding continues
       ... unwinding reached a frame that Unlock the mutex
         Mutex notices it is an incorrect state.
            so it reports the Error
               the Error handler throw an exception .......

and because this exception is being thrown during the unwind, it is fatal.
2020-01-30 08:59:10 +01:00
Chris Burr
500f426c2f Fix building clingutils tetss with builtin_clang=OFF for conda 2020-01-24 15:44:19 +01:00
Vassil Vassilev
0b95ed04a9 Teach cling to handle the module-file-info flag.
The module-file-info action takes a pcm input file, reads the control block
and prints useful information about the module file itself. This is handy when
debugging issues in modules such as header duplications and file relocation.

This patch teaches rootcling to call directly cling and exit via the bare-cling
subcommand.
2019-12-27 13:59:33 +01:00
Pratyush Das
82e662096c Enable tarball-tag flag to build cling using LLVM binary release.
Fixes Cling's nightly releases
2019-12-25 07:29:26 +01:00
Vassil Vassilev
bd0b61528c Allow the atexit functions to be called on their own.
In environments where cling's teardown should be controlled more carefully,
the atexit function handlers should allow to be called separately. This is
useful when a handler depends on alive interpreter and cannot be executed while
the interpreter is half shut down.

This patch should enable TCling's proper shutdown and relax some of the pain
in shutting down ROOT in general.
2019-12-23 19:14:14 +01:00
Vassil Vassilev
ad6a1b47cd Improve shutdown logic.
We should follow the shutdown procedure from FrontendAction::EndSourceFile
which ensures clang is properly torn down.

This patch allows us to write a module file without having to explicitly
call CompilerInstance::clearOutputFiles.
2019-12-23 19:14:14 +01:00