Commit Graph

2425 Commits

Author SHA1 Message Date
Simeon Ehrig
2489cf13b1 Replaced incremental PTX compiler with a internal implementation
Replaced the old version of the PTX compiler which used external tools and llvm::ExecuteAndWait with an internal implementation. The new incremental PTX compiler uses a modified version of the cling::Interpreter instance. The instance can process the PTX built-ins and generates LLVM IR. The LLVM IR is be compiled to PTX via an additional NVPTX backend implemented in the IncrementalCUDADeviceCompiler.

The new implementation has many advantages:
- it's much faster than the old version
- less error-prone because the ASTPrinter and some unnecessary cling transformations are avoided
- reduction of problems in searching for external tools (can be very complicated on HPC systems)

The IncrementalCUDADeviceCompiler is moved from the cling::IncrementalParser to the cling::Interpreter, because the second interpreter needs the input without wrappers or transformations.
2019-11-07 19:29:15 +01:00
Simeon Ehrig
96366346c0 Added DeviceKernelInliner ASTTransformer
This ASTTransformer adds an inline attribute to any CUDA __device__ kernel
that does not have the attribute. Inlining solves a problem caused by
incremental compilation of PTX code. In a normal compiler, all definitions
of __global__ and __device__ kernels are in the same translation unit. In
the incremental compiler, each kernel has its own translation unit. In case
a __global__ kernel uses a __device__ function, this design caused an error.
Instead of generating the PTX code of the __device__ kernel in the same file
as the __global__ kernel, there is only an external declaration of the
__device__ function. However, normal PTX code does not support an external
declaration of functions.

The transformer only works if the target device is nvptx.
2019-11-07 19:29:15 +01:00
Simeon Ehrig
6c46b65754 Extend cling::Intpereter to incremental compile PTX code for CUDA devices
- change CUDA to CUDAHost and add CUDADevice to the InvocationOptions
- in the PTX mode, some ASTTransformer will not be used, which are useful for the x86 mode
2019-11-07 19:29:15 +01:00
Vassil Vassilev
b1c18ae3cf Print the vfs overlay file in the log in verbose mode. 2019-11-01 09:19:38 +01:00
Vassil Vassilev
b2e854cb13 Add an initial version of a modulemap for cuda.
This should fix our TMVA cuda builds.
2019-10-29 12:29:39 +01:00
Vassil Vassilev
d4e822e8d9 Sink {libc,std}.modulemap in cling.
This patch teaches cling to detect if the essential libraries have modulemaps
and if necessary it adds an overlay around libc and std.

This tightens the implementation and makes cling standalone easier to run in
-fmodules mode.
2019-10-22 14:14:49 +02:00
Vassil Vassilev
02f7ae2a91 Collect the modulemaps even if no overlay is set.
This patch tightens the implementation of the collection of modulemap files.
It still gathers all 'system' modulemaps necessary for cling to run if
-fno-implicit-module-maps is specified.

This patch should unbreak our osx builds.
2019-10-21 15:44:51 +02:00
Vassil Vassilev
313734145e Iterate only once. Use early exits. 2019-10-21 15:44:51 +02:00
Vassil Vassilev
7e544b502a Remove negation from parameter name, add default value. NFC. 2019-10-21 15:44:49 +02:00
Vassil Vassilev
024cf290d4 Teach cling about explicit modulemaps; Move TROOT::Get*Dir to FoundationUtils.
If we use -fimplicit-module-maps this means that cling will find all files called
module.modulemap on the include paths. In certain cases the same modulemap can
be present in two locations causing module redefinition errors.

This patch teaches cling to work with explicitly specified modulemaps in
-fno-implicit-module-maps mode. It moves the generation of the overlay file
closer to the CIFactory so that we can reuse the modulemap loading code before
the interpreter object was set up.

The patch also turns off the implicit module map discovery and explicitly
specifies the ROOT-related modulemap files. The modulemap files need to be
enumerated in both TCling and rootcling. Rootcling requires -fno-rtti build mode
and we cannot use the utilities in TROOT (also because rootcling_stage1 is
built before libCore). This requires the extraction of the `GetSysRoot`,
`GetIncludeDir` and `GetEtcDir` in the `ROOT::FoundationUtils` which is can be
used throughout the entire core component.

This fixes ROOT-10354.
2019-10-17 18:59:48 +02:00
Vassil Vassilev
cd0cb53f85 Don't rely on the LD_LIBRARY_PATH for prebuilt modules.
The system integrity protection (SIP) on osx blocks 'dangerous' env variables
when spawning a new process. The MetaProcessor command `.!` which calls the
shell does not propagate (DY)LD_LIBRARY_PATH variables which prevents cling
from finding the modules.

This patch introduces a new env variable CLING_PREBUILT_MODULE_PATH which
contains the prebuilt modules' location where cling should look for modules.

Patch by Alexander Penev!
2019-10-13 08:44:36 +02:00
Bertrand Bellenot
f80d7980aa Change some LangOptions for MSVC (#4503)
* Change some LangOptions for MSVC

Add 'MSVCCompat' flag and set 'ThreadsafeStatics = 0' (this fixes crash when running line.cxx and text.cxx root7 tests in interpreted mode)

* Add a 'FIXME' comment (thanks Axel for the suggestion)
2019-10-11 08:59:19 +02:00
Bertrand Bellenot
e037a02a9a Fix JITted variables on Windows (#3590)
* Fix for JITted variables on Windows

Fixes global, static, and "const int" variables access from experimental PyROOT on Windows

* formatting

* Use mangledName.compare() instead of mangledName.find() and add comments

* Small modification from Wim
2019-10-03 11:51:30 +02:00
Javier Lopez-Gomez
13485246a7 Fixes issue in DeclUnloader: do not unload templates intantiated in the PCH 2019-10-01 01:15:50 +02:00
Javier Lopez-Gomez
e48a3bc179 Disable DefinitionShadower by default; manually enable it from TCling
As requested by vgvassilev, DefinitionShadower is disabled by default in ClinG
and enabled manually in TCling, i.e. usable from the ROOT prompt and Jupyter
notebooks.
2019-09-27 14:59:05 +02:00
Javier Lopez-Gomez
15e67afef2 DefinitionShadower: reuse __cling_N5xxx namespace for decls coming from the same transaction 2019-09-27 14:59:05 +02:00
Javier Lopez-Gomez
93ae61f704 DefinitionShadower: added `DefinitionShadowed' interpreter callback 2019-09-27 14:59:05 +02:00
Javier Lopez-Gomez
2fb847c438 DefinitionShadower: add support for function overload and class/function templates 2019-09-27 14:59:05 +02:00
Javier Lopez-Gomez
1f3b05bd64 DefinitionShadower: modifications suggested in the PR 2019-09-27 14:59:05 +02:00
Javier Lopez-Gomez
8ee98e15c5 Fixes `.undo X' issue (tried to patch wrong lookup table) 2019-09-27 14:59:05 +02:00
Javier Lopez-Gomez
d02353c10b Added new ASTTransformer: DefinitionShadowing. Includes minor changes required to DeclExtractor.
This adds Cling support for function/type/var redefinitions, e.g.
```
int i = 0;
float i = 1.0f;
```
2019-09-27 14:59:04 +02:00
Bertrand Bellenot
e7ae55720e Fix a crash when using an undeclared identifier (Jira #ROOT-10193) 2019-09-24 15:14:12 +02:00
Vassil Vassilev
1a1a84dfcb Ignore import decls to delay the running module inits.
If we preload modules using cling::Interpreter::loadModule the
deserializations of the module initializers (such as global variables).

We want the behavior of clang::Sema::ActOnModuleImport, however, it requires
valid source locations.

This patch ignores the ImportDecls which presumably came from loadModule.
2019-09-20 13:14:06 +02:00
Vassil Vassilev
63e0a871c1 Quote the imported module.
In some cases we have dict.C which the modules system considers as two
components: one top-level module and one submodule.
2019-09-14 22:59:04 +02:00
Vassil Vassilev
d66199f99f Reland "[cxxmodules] Use common-rooted valid source location."
Original commit message:
"Over the years we have a pathological issue with cling when it calls directly
compiler API. Most of the API assume they are called from code residing in a
text file. This code has valid source locations which can be used for
comparisons and things like point of instantiation for template instantiations.

This means that whenever a clang API requires a valid source location, cling
should have an interface which gives a pseudo-valid, commonly-rooted unique
source location.

We are bitten by this fact when preloading modules as if we have diagnostics
they cannot be ordered due to the fact that the compared decls from two
different modules do not have a common includer.

This patch avoids using an API and relies on a textual form which does not have
this problem at the cost of synthesizing an extra string."
2019-09-14 07:59:43 +02:00
Axel Naumann
5446ed5a48 Allow /clang-7" -cc1 to ge found. 2019-09-10 16:29:09 +02:00
Philippe Canal
bd472f9b0a Clang-formatting 2019-09-09 19:15:34 +02:00
Philippe Canal
8b906da196 Expand StartParsingRAII
In particular the CachedTokens need to be pushed/pop or otherwise
give misleading information.

This fixes ROOT-10224.
2019-09-09 19:15:34 +02:00
Vassil Vassilev
1a53d8241d Revert "[cxxmodules] Use common-rooted valid source location."
This reverts commit 03febcbd899610de1188886a132fe78c804d3a00.

This commit broke the modules bots. Investigating...
2019-09-04 15:14:06 +02:00
Vassil Vassilev
b47bbaf546 Use common-rooted valid source location.
Over the years we have a pathological issue with cling when it calls directly
compiler API. Most of the API assume they are called from code residing in a
text file. This code has valid source locations which can be used for
comparisons and things like point of instantiation for template instantiations.

This means that whenever a clang API requires a valid source location, cling
should have an interface which gives a pseudo-valid, commonly-rooted unique
source location.

We are bitten by this fact when preloading modules as if we have diagnostics
they cannot be ordered due to the fact that the compared decls from two
different modules do not have a common includer.

This patch avoids using an API and relies on a textual form which does not have
this problem at the cost of synthesizing an extra string.
2019-09-04 12:44:07 +02:00
Axel Naumann
6a73fd08d2 Help ValuePrinter find the transaction:
If the ValuePrinter runtime header gets included,
a transaction might be emitted, and the transaction
containing the function body cannot be found anymore.
This causes roottest/root/meta/getFuncBody.C to fail
with runtime modules.
2019-09-01 09:15:17 +02:00
Javier Lopez-Gomez
a829d76c10 Fix SIGSEGV after executing the `.stats decl' metacommand 2019-08-16 16:59:04 +02:00
Guilherme Amadio
0d3fb9dbf4 Simplify query for include directories
- Use only sed instead of awk and grep
- Use regex independent of locale settings
2019-08-14 14:59:45 +02:00
Raphael Isemann
8446b953c2 Rename stl module to std
libc++ comes with a builtin modulemap and is using "std" as the
module name for the STL. To stay consistent we should use the same
name (and "std" is anyway a better name).
2019-07-25 17:59:15 +02:00
Vassil Vassilev
a63b8f1ec7 Mark library search paths system and user accordingly.
We have user library search paths which come from LD_LIBRARY_PATH or similar;
and system library search paths which are coming from the platform.

This patch enables external users performing symbol resolution to filter out
system search paths when they know the symbol cannot be there.

Now, TClingCallbacks can merge the modules vs non-modules code paths, where
the modules prebuilt path is essentially our LD_LIBRARY_PATH.
2019-07-18 20:59:17 +02:00
Axel Naumann
a3f236fe36 Use canary bytes to determine whether to run dtor:
If the constructor of the contained object is not run (e.g. because
assembling its arguments triggered an exception), the dtor must not
be run when destructing the cling::Value. Detect this case by imprinting
canary bytes into the contained object bytes: if they have changed,
run the dtor, if not assume that the constructor has failed.

This will cause false positives in those cases where the constructor
is not modifying the first object bytes: in these cases, the dtor
is not run even though the ctor is run. That is still better than
the other case (where the dtor crashes because no ctor was run).
2019-07-11 14:44:14 +02:00
Vassil Vassilev
da626b392d Teach ACLiC to build modules.
ACLiC now synthesizes a modulemap with a suffix _ACLiC_dict.modulemap. The file
contains the source file to be compiled and the corresponding library.

The modulemap is then passed to rootcling via -fmodule-map-file= flag to avoid
naming clashes with possibly existing other modulemap files.

This patch teaches cling to work with the -fmodule-map-file= flag.

ACLiC supports automatic inclusion of Rtypes.h (making ClassDef macro
available). Modules are built in isolation and are resilient to #include
of Rtypes.h at rootcling startup time. We make module Core (containing Rtypes.h)
visible via a newly implemented callback.
2019-07-01 13:59:14 +02:00
Axel Naumann
70df14a1c7 Create transactions only if codegen (not rootcling):
The transactions are useful to buffer Decls before sending them to CodeGen,
to verify semantic validity before emitting. If there is no codegen,
DeclCollecting them wastes CPU cycles during PCH generation time
because all headers are in one single transaction. Of course this also
speeds up regular rootcling invocations.
2019-06-28 23:59:17 +02:00
Axel Naumann
8f20c8c6a1 Remove unnecessary include (NFC). 2019-06-28 23:59:17 +02:00
Axel Naumann
ab8fcc633a Remove default-by-tokens default ctor (NFC). 2019-06-28 23:59:17 +02:00
Vassil Vassilev
74685d3725 Store rdict files as module file extensions.
This patch moves the ROOT-specific rdict.pcm optimization in the EXTENSION_BLOCK
of a C++ module file.

This reduces the generated artifacts and simplifies the loading of a rdict pcm
file as it is now part of the C++ module file. This patch paves our way to
using the global module indexing.
2019-06-19 18:59:07 +02:00
Philippe Canal
ddffe49bea Insure that the commit callbacks don't change the 'current' transaction.
This should solve the problem we have been seeing with cmsUnload.  The problem stems from the fact that TCling::AutoParseImplRecurse
uses the address of the current transactions has an index/key when registering a class in fTransactionHeadersMap.

On some platforms TInterpreter.h is parsed late, i.e. during the compilation of:

   gInterpreter->AutoParse("SiStripCluster");

consequently (as AutoParse does not create its own transaction), the transactions that contains
the parsin of TInterpreter is the last one committed before AutoParseImplRecurse is executed.

Without the parsing of TInterpreter.h then that transaction is return by fInterpreter->getCurrentTransaction().
With the parsing of TInterpreter.h, during the commit of the transaction of the callbacks are triggered
and one of the decl is a constant (kFALSE) and thus, in TCling::HandleNewDecl, provokes the execution of

       if (gROOT->GetListOfGlobals()->FindObject(ND->getNameAsString().c_str()))
         return;

which will triggered the creation of a Transaction, however since the last top level transaction was
just switch to the state 'Committed', the new transaction will not be nested and thus
IncrementalParser::endTransaction will set the current transaction to 'nullptr'.

And this value of nullptr for the key of fTransactionHeadersMap is neither guaranteed to be unique
nor handled properly by the rest of the code.

It is unclear why this problem have surfaced recently.

On a failing machine, rebuilding with a 2 month old commit (ad9e5c42cb) still exhibits the problem.
2019-06-07 08:29:45 +02:00
Vassil Vassilev
81e2eda009 Check if the triggerFunc is in a library.
This patch should fix an assert in the incrementals.
2019-06-03 15:44:13 +02:00
Vassil Vassilev
d71643dcf8 Support recursive lookup helper calls.
The implementation of class->library mapping makes a call to the
LookupHelper::findScope. This makes the recursive invocations to
LookupHelper::findScope -> ... -> LookupHelper::findScope happen more often.
2019-06-02 21:44:26 +02:00
Vassil Vassilev
03c9a184ba Do not copy the FrontendOpts, we want to modify the object. 2019-05-14 08:47:27 +02:00
xloem
8acc94bfd6 Protect possibly predefined preprocessor symbol
On my systems (RHEL7 and Android Termux) PAGE_SIZE is defined to 4096 in /usr/include/sys/user.h .  The enum then produces a compile-term error.
Adding this change resolves this error for me.
2019-05-13 15:01:11 +02:00
Vassil Vassilev
ea23e15d42 Remove workaround used for ROOT. 2019-04-22 17:14:37 +02:00
Vassil Vassilev
fb5a493198 Resize the memory cache before entering the loop.
This prevents the vector to be reallocated if the size exceeds the capacity of
the vector causing invalid pointer accesses.

Fixes ROOT-7749.
2019-04-09 15:29:12 +02:00
Vassil Vassilev
27f03abc6c Typo. 2019-03-21 23:29:11 +01:00
Vassil Vassilev
e7b7b6c247 OSX does not require overlay files. 2019-03-21 23:29:11 +01:00