IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Prior to the upgrade to LLVM13, child interpreters used to also lookup symbols
in their parent.
This commit introduces a `DefinitionGenerator` that allows for symbol lookup
across different `IncrementalJIT` instances, which restores the old behavior.
This change fixes the following tests:
- CodeUnloading/AtExit.C
- MultipleInterpreters/MultipleInterpreters.C
Fixes#12455.
With the upgrade to llvm-13, the JIT lost the ability to re-use existing weak
symbols that the JIT had already emitted, instead only looking at dlsym. This
causes a significant increase in JITted symbols, and thus a significant slow-down
of cling / its JIT.
This restores the old behavior, with an identical set of symbols that jet jitted.
With the upgrade, BackendPasses was modifying a TargetMachine that was
not used by SimpleCompiler.
Change that by
- using a SimpleCompiler that uses IncrementalJIT::TM;
- moving the TM creation to IncrementalJIT, and giving access to it
This reduces the runtime of https://github.com/root-project/root/issues/11927
to
- before llvm upgrade: 2.69s
- llvm13, without this commit: ???
- llvm13, with this commit: 2.89s
i.e, a slow-down of 7% (that is likely caused by the different emission
mechanism of Orc-v2; to be confirmed...)
OrcV2 uses specific DyLibs to resolve symbols from, we need a dedicated
resolver for libCling because its symbols cannot be found from the
process.
Remove now unused "ExposeHiddenSharedLibrarySymbols".
Replace the existing LazyFunctionCreator interface by a function
to add DefinitionGenerators to be passed via the chain Interpreter
-> IncrementalExecutor -> IncrementalJIT.
Implement a DefinitionGenerator in TCling(Callbacks) to define
MaterializationUnits for autoloading symbols. This also allows to
remove the double DynamicLibrarySearchGenerator now that the created
AutoloadLibraryMUs inject the addresses into the JIT after loading
the required library.
IncrementalExecutor has its own, showing what is requesting the
symbols, which is more useful than the diagnostic of Orc. OTOH
Orc might emit more errors than just missing symbols, and if
there are multiple missing symbols, IncrementalExecutor only
shows the first.
So enable `Verbose` also for IncrementalJIT, showing the original
missing symbols as diagnosed by Orc.
This fixes roottest_cling_other_checkMissingSymbolExitCode which
fails due to the extra diagnostic error line.
OSX (due to legacy to disambigate between asm and C symbols) implements an extra
linker-level mangling which adds another `_` in the name. However, that is only
in the on-disk representation and needs to be dropped upon calling dlsym.
OrcV2 gives us a handle to the symbol with starts with `__` allowing us to
invert the logic of the symbol search where we drop the `__` for dlsym.
This patch also inverts the parameter from ExcludeHostSymbols to
IncludeHostSymbols to improve readability.
As written in one of the FIXMEs this is a temporary solution which allows us to
go further with the llvm-13 upgrade. We can rework the getAddress* functionality
to reuse more the ORCV2 infrastructure and more importantly to use that
infrastructure in the recommended way.
When removing code, we need to recover Clang's CodeGen and this happens by
iterating over the clang::Decls and the llvm::Module for a given Transaction in
the TransactionUnloader.
The problem is that now OrcV2 takes the ownership of the llvm::Module from the
Transaction for itself and then it can notify us when the module was compiled on
demand. Unfortuantely, there is no good way to guarantee that the same module
will be attached to the same transaction. This approach allows the
TransactionUnloader::unloadModule logic to recover CodeGen.
In a longer term, when we switch to partial translation units from clang-repl
we might be able to recover CodeGen without needing to know details about the
generated llvm::Module.
This change was from MCJIT times and now is not needed anymore. Moreover, the
orcv2 jit infrastructure considers the llvm::Module immutable after it takes
control of it via emitModule. This change will allow us to migrate easier to
orcv2.
Instead of suppressing the emission of weak symbols that have an existing
in-process definition, remove the definition of the weak symbol from the
module. This prevents relocation "surprises" as seen e.g. on Apple M1, where
weak symbol definitions are expected to be found within the module, and
relocations are thus expected to be local. By removing the definition, the
symbol becomes a reference (declaration) and relocation will work as for any
non-local symbol.
Thus also reduces the amount of duplicate work, as in-process weak symbols
will now be suppressed early.
The dynamic library manager's dyld can search a symbol in the library path.
This patch uses that functionality to aid the 'unresolved while linking'
diagnostics.
Now we get:
cling -L lib/
****************** CLING ******************
* Type C++ code and press enter to run it *
* Type .q to exit *
*******************************************
[cling]$ extern int gErrorIgnoreLevel;
[cling]$ gErrorIgnoreLevel
IncrementalExecutor::executeFunction: symbol 'gErrorIgnoreLevel' unresolved while linking [cling interface function]!
Symbol found in '/.../lib/libCore.so'; did you mean to load it with .L /.../lib/libCore.so ?
[cling]$
In tests on an Apple M1 after the upgrade to LLVM 9, this new
instruction selection framework emits branches / calls that expect
all code to be reachable in +/- 128 MB. This cannot be guaranteed
during JIT, which generates code into allocated pages on the heap
and could span the entire address space of the process.
The old JIT infrastructure assumed shared ownership of the llvm::Module
implemented via shared_ptr. This guaranteed each client can have a uniform
view of the llvm::Module. The new infrastructure claims ownership of the object
and even more transfer ownership when the llvm::Module travels through different
layers of the JIT. The claimed advantage is better thread safety.
The new logic defines away an important property which cling has been built
around, that is the shared symbol object ownership.
This patch makes the cling::Transaction the owner of the llvm::Module. The
ownership is transfered when we want the JIT to 'emit' the module. Fortunately,
there is a JIT callback which can transfer back the ownership to the transaction.
This preserves some consistency, however, makes some operations unsafe. For
example, we cannot rely on Transaction::getModule when the module is being
handed to the JIT.
This patch tries to adapt to the new infrastructure by reducing the dependency
on the transaction's llvm::Module in favor of relying more on the information
available in the execution engine (eg. at_exit handling).
In environments where cling's teardown should be controlled more carefully,
the atexit function handlers should allow to be called separately. This is
useful when a handler depends on alive interpreter and cannot be executed while
the interpreter is half shut down.
This patch should enable TCling's proper shutdown and relax some of the pain
in shutting down ROOT in general.
The user might use utilities which print on cout and expects the output
to be shown immediately.
This patch automatically flushes std::cout after each execution of a wrapper.
This patch contains two functionality:
1. Autoloading of dynamic symbols for system headers
There is three kind of symbols in shared object file, which is 1
normal symbols, 2 dynamic symbols, and 3 hidden visibility symbols.
Linker doesn't care about 3, but should take care (of course) 1 and
2. For system headers, often symbols are defined in .dynsym section
which means they are 2 dynamic symbols. This patch adds support of
autoloading those symbols. We fallback to resolving dynamic symbols
from system headers only if we couldn't resolve from normal symbol
table, as the initialization of header search is expensive (iterating
through all system headers)
2. Register callback from IncrementalExecutor
Previously, LazyFunctionCreatorAutoload was getting callback only
from DynamicLibraryManager::loadLibrary. This was enough for fixing
tests, but is insufficient to handle "symbol <something> unresolved
while linking function" errors as those errors are emitted from
IncrementalExecutor. Adding a callback from IncrementalExecutor
enables us to unresolved symbols.
It fixes these kind of errors:
`IncrementalExecutor::executeFunction: symbol '_ZN7TCanvasC1EPKcS1_iiii' unresolved while linking function '_GLOBAL__sub_I_cling_module_8'!`
TLS is currently not suppored in the JIT on some platforms.
However, it's possible to enable emulated TLS support in LLVM
which means that we now support TLS across many architectures.
The performance downsides of this are the overhead of accessing
the variable due to the additional indirection by the emulation.
However, this overhead is minimal and shouldn't affect most
programs. It also can be easily worked around from the user side.
This can be donefFor example by wrapping TLS variables into a single
TLS struct variable that then contains the other variables. Or just
minimizing referencing the TLS variable and use a normal copy of
the variable instead and write it back later.
Patch created with a lot of help from Lang Hames and Pavel Labath!
This is in prepare for the upcoming llvm upgrade. The future orc jit compile
layer needs a std::shared_ptr<llvm::Module>. The current design passes a
llvm::Module* around and any conversions to a shared_ptr cause the
destruction of the llvm::Module which is a long-lived object in cling.
