Commit Graph

3305 Commits

Author SHA1 Message Date
ferdymercury
9cdbf1d0eb nullptr use
clingUtils modernize nullptr
2023-01-26 08:59:06 +01:00
Jiang Yi
90a7639461 Support setting cmd history file by ${CLING_HISTFILE} 2023-01-25 13:59:06 +01:00
Jiang Yi
392d5365d5 Set history size to ${CLING_HISTSIZE}
It was left unset. Then the default hist size was

static_cast<size_t>(-1), which is practically equivalent to unlimited

history size.
2023-01-25 13:59:06 +01:00
Jonas Hahnfeld
d47b49c3ee Reset function sections before JITting (#12069)
This makes all functions end up in the same text section, which is
important for TCling on macOS to catch exceptions from constructors:
Stack unwinding requires information about program addresses to find
out which objects to destroy and what code should be called to handle
the exception. These addresses are relocated against a single __text
section when loading the produced MachO binary, which breaks if the
call sites of global constructors end up in a separate init section.

Fixes ROOT-10703 and ROOT-10962

(cherry picked from commit 028fcca0fa76111877751df876cf13968be602f9)
2023-01-24 10:29:05 +01:00
Jonas Hahnfeld
2f48c65752 Explicitly create FunctionDecl in DeclExtractor
The function EnforceInitOrder() was using ImplicitlyDefineFunction()
as a shortcut to define a function returning an int. Future upgrades
of LLVM will add an assert in that function because it is not allowed
to be used when compiling C++, which Cling obviously does.
2023-01-23 18:44:07 +01:00
Jonas Hahnfeld
e1ae677300 Fix in-process lookup of symbols without GlobalPrefix
On Windows, C++ symbols are prefixed not with the "global" prefix,
but with a question mark.
2023-01-20 08:44:02 +01:00
Jonas Hahnfeld
10912bdab3 Remove explicit setting of COFF binary format on Windows
This is the default anyway.
2023-01-20 08:44:02 +01:00
Axel Naumann
feb0db382e For MSVC weak symbols, re-use even small funcs:
Without, the JIT fails to compile roottest/root/aclic/misc/assertROOT7027.C
correctly.
2023-01-17 16:59:06 +01:00
Axel Naumann
8a87d1f8d6 JIT: find existing weak symbol without materializer:
m_JIT.getSymbolAddress() invokes the symbol materializers, which compile (which
is sort of okay) but also try autoloading (which totally is not okay).

Instead, implement a function to search existing JIT symbols.

This can be accelerated by looking up the whole set of symbols, instead of doing it
symbol by symbol. I leave that refactoring for later...
2023-01-17 16:59:06 +01:00
Axel Naumann
1b6158779e JIT: separate re-use of weak sym vars, funcs (NFCI). 2023-01-17 16:59:06 +01:00
Axel Naumann
2925ee3a32 JIT: only re-use *large* existing weak functions:
Small functions might get inlined, and hiding their definition prevents
the inliner from doing its job.
2023-01-17 16:59:06 +01:00
Axel Naumann
4b2e5f8e2c Reuse existing weak symbols, also from JIT:
With the upgrade to llvm-13, the JIT lost the ability to re-use existing weak
symbols that the JIT had already emitted, instead only looking at dlsym. This
causes a significant increase in JITted symbols, and thus a significant slow-down
of cling / its JIT.

This restores the old behavior, with an identical set of symbols that jet jitted.
2023-01-17 16:59:06 +01:00
Axel Naumann
da247bd77a Re-enable CodeGen-OptLevel after llvm13 upgrade:
With the upgrade, BackendPasses was modifying a TargetMachine that was
not used by SimpleCompiler.

Change that by
- using a SimpleCompiler that uses IncrementalJIT::TM;
- moving the TM creation to IncrementalJIT, and giving access to it

This reduces the runtime of https://github.com/root-project/root/issues/11927
to
- before llvm upgrade: 2.69s
- llvm13, without this commit: ???
- llvm13, with this commit: 2.89s
i.e, a slow-down of 7% (that is likely caused by the different emission
mechanism of Orc-v2; to be confirmed...)
2022-12-21 17:14:02 +01:00
Vassil Vassilev
cefa80ef54 Rebase better to llvm13 to the FileEntryRef and FileEntry.
Likely fixes root-project/root#11933
2022-12-20 08:59:05 +01:00
Guilherme Amadio
b0940d1ba7 Re-enable support for profiling/debugging interpreted/JITted code
This feature, originally added in commit 22b1606f, was reverted to
make the LLVM upgrade to version 13 easier. This commit adds back
all functionality as it was just before the LLVM upgrade.
2022-12-15 10:29:03 +01:00
Jonas Hahnfeld
b9ec7687bf Create non-empty source file
LLVM currently has an issue that a completely empty source file is
stored as a nullptr, but DIFile::getSource() does not correctly deal
with this situation. A crash can be observed if just attempting to
launch root.exe with EXTRA_CLING_ARGS="-gdwarf-5 -gembed-source".

See https://reviews.llvm.org/D137152 and https://reviews.llvm.org/D138658
for discussions with upstream.
2022-12-13 09:59:05 +01:00
Jonas Hahnfeld
092c9c981f Increase initial offset into virtual file
Otherwise the PresumedLoc's suddely get valid once we avoid a
completely empty soure buffer in the next commit.
2022-12-13 09:59:05 +01:00
Bertrand Bellenot
9cfaafc10e Fix compilation with VS 2022 v17.4 (#11866)
* Fix compilation with VS 2022 v17.4

This fixes the issue #10875 _HAS_CONDITIONAL_EXPLICIT=0 won't work with VS 2022 17.4

* In fact with LLVM 13 this workaround is not needed anymore
2022-12-12 11:29:05 +01:00
Axel Naumann
7aac81b4c0 Fix bug in handling EXTRA_CLING_ARGS:
The splitting code requires the full string to be null-terminated.
Before, random bytes were attached to the last option.
2022-12-09 08:44:20 +01:00
Axel Naumann
b0cb236016 Track (and unload) symbols from dylibs:
When creating orec-Symbols for dylib symbols, reloading the
dylib might mean a change in symbol (address). So unloading a
dylib means we need to unload the orc-Symbol.

This is implemented through resource-tracking the symbols as
provided by DynamicLibrarySearchGenerator. Actually, as
DynamicLibrarySearchGenerator does not support resource tracking,
it is implemented in a near-copy of DynamicLibrarySearchGenerator,
RTDynamicLibrarySearchGenerator, which uses the transaction of the
most recent module for the ResourceTracker.
2022-12-09 08:44:20 +01:00
Axel Naumann
8a267f84e3 Remove unused data members. 2022-12-09 08:44:20 +01:00
Axel Naumann
0950faa211 Remove spurrious string generation. 2022-12-09 08:44:20 +01:00
Jonas Hahnfeld
1f237a50ee Disable release of code and data sections
This is required until CallFunc is informed about unloading, and can
re-generate the wrapper (if the decl is still available).
2022-12-09 08:44:20 +01:00
Jonas Hahnfeld
4f3af1dc2c Reintroduce custom memory manager
This is still required for proper exception handling support. See
commits 3f74182697 and a7b0b3e647 by Philippe for details on the
problem and the original code, which I was able to significantly
simplify with the new JIT infrastructure.

For the moment, this disables JITLink even on platforms that had it
active by default (notably macOS). This will be reintroduced at a
later point, which will require a memory manager implementation for
the JITLink interface.
2022-12-09 08:44:20 +01:00
Axel Naumann
13b4be1603 Make IncrementalExecutor::replaceSymbol() take IR names:
Previously, one needed to pass linker-mangled names, which exposes details that
clients of IncrementalExecutor should not have to deal with. Instead, use the IR
name and do the linker-mangling in IncrementalJIT::addOrReplaceDefinition().

This fixes the lack of static destruction on macOS, visible e.g. in the test
failure of roottest/cling/staticinit/ROOT-7775.
2022-12-09 08:44:20 +01:00
Jonas Hahnfeld
76d96e317f Mark __dso_handle a "far" pointer:
The relocation needs to allow for long offsets, as for the JIT, __dso_handle
might be outside the shared library. Fixes
```
cling JIT session error: In graph cling-module-10-jitted-objectbuffer, section __TEXT,__StaticInit: relocation target "___dso_handle" at address 0x7fe1ee5052e0 is out of range of Delta32 fixup at 0x108c410bd (___cxx_global_var_initcling_module_10_, 0x108c41090 + 0x2d)
[runStaticInitializersOnce]: Failed to materialize symbols: { (main, { $.cling-module-10.__inits.0, __ZN12IncidentTypeL2m1E, __ZN6MarkerD2Ev, __ZN6MarkerD1Ev, ___cxx_global_var_initcling_module_10_.1, __GLOBAL__sub_I_cling_module_10, __ZN6MarkerC2EPKc, ___cxx_global_var_initcling_module_10_.3, __ZN12IncidentTypeL2m3E, __ZN6MarkerC1EPKc, __ZN12IncidentTypeL2m2E, ____orc_init_func.cling-module-10, ___cxx_global_var_initcling_module_10_ }) }
```
as seen no RISC-V and macOS, i.e. with the JITLinker.
2022-12-09 08:44:20 +01:00
Vassil Vassilev
89543f66cf Add an absolute symbol pointing to _CxxThrowException on Windows 2022-12-09 08:44:20 +01:00
Axel Naumann
21f2c66a0f Delete globals before JIT deletes the module:
Fixes root-meta-execUnloading which shows
```==768591== Invalid read of size 8
==768591==    at 0x6EFB10E: llvm::ilist_node_base<false>::getNext() const (ilist_node_base.h:29)
==768591==    by 0x6FBF119: llvm::ilist_node_impl<llvm::ilist_detail::node_options<llvm::Function, false, false, void> >::getNext() (ilist_node.h:66)
==768591==    by 0x6FBED48: llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>::operator++() (ilist_iterator.h:157)
==768591==    by 0x6FBEA20: llvm::simple_ilist<llvm::Function>::begin() (simple_ilist.h:117)
==768591==    by 0x6FBE40F: llvm::Module::begin() (Module.h:622)
==768591==    by 0x6FBE475: llvm::Module::functions() (Module.h:634)
==768591==    by 0x6FBE22F: cling::TransactionUnloader::unloadModule(llvm::Module*) (TransactionUnloader.cpp:175)
==768591==    by 0x6FBDF83: cling::TransactionUnloader::RevertTransaction(cling::Transaction*) (TransactionUnloader.cpp:126)
==768591==    by 0x6F90CD0: cling::Interpreter::unload(cling::Transaction&) (Interpreter.cpp:1553)
==768591==    by 0x6F90E25: cling::Interpreter::unload(unsigned int) (Interpreter.cpp:1574)
==768591==    by 0x721A9F2: cling::MetaSema::actOnUndoCommand(unsigned int) (MetaSema.cpp:186)
```
...
```
==768591==  Address 0x144b0ce0 is 32 bytes inside a block of size 800 free'd
==768591==    at 0x484399B: operator delete(void*, unsigned long) (vg_replace_malloc.c:935)
==768591==    by 0x6FBC7CA: std::default_delete<llvm::Module>::operator()(llvm::Module*) const (unique_ptr.h:95)
==768591==    by 0x6FBC82D: std::__uniq_ptr_impl<llvm::Module, std::default_delete<llvm::Module> >::reset(llvm::Module*) (unique_ptr.h:203)
==768591==    by 0x6FBC87E: std::unique_ptr<llvm::Module, std::default_delete<llvm::Module> >::reset(llvm::Module*) (unique_ptr.h:501)
==768591==    by 0x6FBC04E: std::unique_ptr<llvm::Module, std::default_delete<llvm::Module> >::operator=(decltype(nullptr)) (unique_ptr.h:434)
==768591==    by 0x70E60E7: llvm::orc::ThreadSafeModule::~ThreadSafeModule() (ThreadSafeModule.h:116)
==768591==    by 0x936EC79: llvm::orc::IRMaterializationUnit::~IRMaterializationUnit() (Layer.h:31)
==768591==    by 0x93707F5: llvm::orc::BasicIRLayerMaterializationUnit::~BasicIRLayerMaterializationUnit() (Layer.h:121)
==768591==    by 0x9370811: llvm::orc::BasicIRLayerMaterializationUnit::~BasicIRLayerMaterializationUnit() (Layer.h:121)
==768591==    by 0x6E91B45: std::default_delete<llvm::orc::MaterializationUnit>::operator()(llvm::orc::MaterializationUnit*) const (unique_ptr.h:95)
==768591==    by 0x6E8F9A3: std::unique_ptr<llvm::orc::MaterializationUnit, std::default_delete<llvm::orc::MaterializationUnit> >::~unique_ptr() (unique_ptr.h:396)
==768591==    by 0x935D3C3: llvm::orc::JITDylib::UnmaterializedInfo::~UnmaterializedInfo() (Core.h:1062)
==768591==    by 0x935D3DE: void std::_Destroy<llvm::orc::JITDylib::UnmaterializedInfo>(llvm::orc::JITDylib::UnmaterializedInfo*) (stl_construct.h:151)
==768591==    by 0x935D26B: void std::allocator_traits<std::allocator<void> >::destroy<llvm::orc::JITDylib::UnmaterializedInfo>(std::allocator<void>&, llvm::orc::JITDylib::UnmaterializedInfo*) (alloc_traits.h:648)
==768591==    by 0x935CBA0: std::_Sp_counted_ptr_inplace<llvm::orc::JITDylib::UnmaterializedInfo, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:613)
==768591==    by 0x6C39FCE: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:346)
==768591==    by 0x6C3CB36: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:1071)
==768591==    by 0x6E98CED: std::__shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1524)
==768591==    by 0x6E98D09: std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo>::~shared_ptr() (shared_ptr.h:175)
==768591==    by 0x932D614: llvm::DenseMapBase<llvm::DenseMap<llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo>, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo> > >, llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo>, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo> > >::erase(llvm::orc::SymbolStringPtr const&) (DenseMap.h:307)
==768591==    by 0x931028D: llvm::orc::JITDylib::removeTracker(llvm::orc::ResourceTracker&) (Core.cpp:1510)
==768591==    by 0x9315060: llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&)::{lambda()#1}::operator()() const (Core.cpp:2155)
==768591==    by 0x93150D0: decltype(auto) llvm::orc::ExecutionSession::runSessionLocked<llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&)::{lambda()#1}>(llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&)::{lambda()#1}&&) (Core.h:1326)
==768591==    by 0x9315272: llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&) (Core.cpp:2152)
==768591==    by 0x93060FD: llvm::orc::ResourceTracker::remove() (Core.cpp:53)
==768591==    by 0x70F1D58: cling::IncrementalJIT::removeModule(cling::Transaction const&) (IncrementalJIT.cpp:151)
==768591==    by 0x6FBE6CF: cling::IncrementalExecutor::unloadModule(cling::Transaction const&) const (IncrementalExecutor.h:180)
==768591==    by 0x6FBDE5B: cling::TransactionUnloader::RevertTransaction(cling::Transaction*) (TransactionUnloader.cpp:119)
==768591==    by 0x6F90CD0: cling::Interpreter::unload(cling::Transaction&) (Interpreter.cpp:1553)
==768591==    by 0x6F90E25: cling::Interpreter::unload(unsigned int) (Interpreter.cpp:1574)
==768591==    by 0x721A9F2: cling::MetaSema::actOnUndoCommand(unsigned int) (MetaSema.cpp:186)
```
2022-12-09 08:44:20 +01:00
Axel Naumann
3675713fee Remove unused data members, parameters, includes.
(cherry picked from commit 872d7c14d8c8e2883ed20a0b6c66a4bbafa4237d)
2022-12-09 08:44:20 +01:00
Axel Naumann
a319dbff2d Formatting (80cols). 2022-12-09 08:44:20 +01:00
Axel Naumann
d8c5c2a206 Pass libCling handle at construction time, resolve from it:
OrcV2 uses specific DyLibs to resolve symbols from, we need a dedicated
resolver for libCling because its symbols cannot be found from the
process.

Remove now unused "ExposeHiddenSharedLibrarySymbols".
2022-12-09 08:44:20 +01:00
Axel Naumann
0362d3fcda Make in-process sym lookup configurable. 2022-12-09 08:44:20 +01:00
Axel Naumann
7347463577 weak symbol re-emission veto: only consider in-process symbols:
Asking for JIT symbols triggers autoloading. JIT symbols will be
resolved by the JIT itself.
2022-12-09 08:44:20 +01:00
Vassil Vassilev
c11799bb99 Explicitly instantiate pending instantiations.
See llvm/llvm-project@4f3e3811d1
2022-12-09 08:44:20 +01:00
Simeon Ehrig
0ee3ebcb5b Fixed bug in symbol linking for CUDA kernel registration with LLVM 13.
Since cling was ported to LLVM 13, it is required that the
`__cuda_register_globals` function and the `__cuda_fatbin_wrapper` and
`__cuda_gpubin_handle` variables are unique when defining a CUDA kernel.
Otherwise, the JIT is lazy and reuses the compiled version of
`__cuda_register_globals`, `__cuda_fatbin_wrapper` and `__cuda_gpubin_handle`
from the first CUDA kernel definition for all subsequent CUDA kernel
definitions, which in practice means that the PTX code from the first kernel is
re-registered each time.

Increase the default CUDA SM level to 35 because SM 20 is deprecated or
removed in the current CUDA SDK versions.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
927f761b8b TCling: Re-implement autoload via MaterializationUnits
Replace the existing LazyFunctionCreator interface by a function
to add DefinitionGenerators to be passed via the chain Interpreter
-> IncrementalExecutor -> IncrementalJIT.
Implement a DefinitionGenerator in TCling(Callbacks) to define
MaterializationUnits for autoloading symbols. This also allows to
remove the double DynamicLibrarySearchGenerator now that the created
AutoloadLibraryMUs inject the addresses into the JIT after loading
the required library.
2022-12-09 08:44:19 +01:00
Axel Naumann
619f5e83a2 Silence Orc missing symbols message:
IncrementalExecutor has its own, showing what is requesting the
symbols, which is more useful than the diagnostic of Orc. OTOH
Orc might emit more errors than just missing symbols, and if
there are multiple missing symbols, IncrementalExecutor only
shows the first.

So enable `Verbose` also for IncrementalJIT, showing the original
missing symbols as diagnosed by Orc.

This fixes roottest_cling_other_checkMissingSymbolExitCode which
fails due to the extra diagnostic error line.
2022-12-09 08:44:19 +01:00
Axel Naumann
c28fc71858 Remove duplicate / impossible condition. 2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
f63fdda04e Weaken definitions of typeinfos and vtables
Introduce a backend pass that changes the linkage of external symbols
starting with "_ZT" (ie. typeinfo names, typeinfos, and vtables) to
weak symbols. This avoids duplicate symbol errors with the new JIT
linker.

FIXME: This is a hack, we should teach the frontend to emit these
only once, or mark all duplicates as available_externally (if that
improves performance due to optimizations).
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
d8451f6822 Print default template arguments
Fixes (at least) roottest-cling-stl-default-VectorSort and
roottest-cling-templateMembers-runoperatorEqual.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
3bbcfe34b1 Ignore duplicate kept GlobalValues
Set WeakAnyLinkage instead of ExternalLinkage to tell the JIT linker
that it should not error on duplicate symbols.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
b721467e15 Fix emission of vtables
It is (again) necessary to call DefineUsedVTables() to enable (at
least) the emission of implicitly defined destructors overriding
the virtual destructor in a base class, see the added test.
2022-12-09 08:44:19 +01:00
Vassil Vassilev
ed72130ee6 Fix the special case for Windows. FIXE: Try remove ifdef 2022-12-09 08:44:19 +01:00
Vassil Vassilev
b4b96a6d85 The JIT now is more consistent on OSX with the linker-level mangling
OSX (due to legacy to disambigate between asm and C symbols) implements an extra
linker-level mangling which adds another `_` in the name. However, that is only
in the on-disk representation and needs to be dropped upon calling dlsym.

OrcV2 gives us a handle to the symbol with starts with `__` allowing us to
invert the logic of the symbol search where we drop the `__` for dlsym.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
b7aaa96719 Print 'a<b<c> >' rather than 'a<b<c>>' in ValuePrinter
See earlier commit for the Transform.C test.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
ae776451fc Move Autoload annotation after scope keyword
Otherwise tests/Autoloading/AutoForwarding.C complains that "an
attribute list cannot appear here".
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
106e1771aa Fix some trivial compiler warnings
"moving a local object in a return statement prevents copy elision" as
well as an unused variable and function referenced only from an assert.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
1cba6d415c Do not externalize unnamed GlobalValues
It doesn't make sense and the JIT crashes, see upstream bug report
https://github.com/llvm/llvm-project/issues/54813
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
b20a403638 Fix CUDA architecture selection
--cuda-gpu-arch aliases --offload-arch, so the option parser will
canonicalize to the latter.
2022-12-09 08:44:19 +01:00
Vassil Vassilev
b9ac132337 Try fixing the multicore 2022-12-09 08:44:19 +01:00