Commit Graph

5105 Commits

Author SHA1 Message Date
Jiang Yi
90a7639461 Support setting cmd history file by ${CLING_HISTFILE} 2023-01-25 13:59:06 +01:00
Jiang Yi
392d5365d5 Set history size to ${CLING_HISTSIZE}
It was left unset. Then the default hist size was

static_cast<size_t>(-1), which is practically equivalent to unlimited

history size.
2023-01-25 13:59:06 +01:00
Jonas Hahnfeld
d47b49c3ee Reset function sections before JITting (#12069)
This makes all functions end up in the same text section, which is
important for TCling on macOS to catch exceptions from constructors:
Stack unwinding requires information about program addresses to find
out which objects to destroy and what code should be called to handle
the exception. These addresses are relocated against a single __text
section when loading the produced MachO binary, which breaks if the
call sites of global constructors end up in a separate init section.

Fixes ROOT-10703 and ROOT-10962

(cherry picked from commit 028fcca0fa76111877751df876cf13968be602f9)
2023-01-24 10:29:05 +01:00
Vassil Vassilev
2949bf96b7 Bump clad version to v1.1.
This new release includes some improvements:
  * Improvements in the array handling in the Error Estimation Framework
  * Fixed numerical stability of the pow pushforward
  * Fixed handling of for-loop conditions in reverse mode
  * LLVM15 Support

See more at: https://github.com/vgvassilev/clad/blob/v1.1/docs/internalDocs/ReleaseNotes.md
2023-01-23 19:59:07 +01:00
Jonas Hahnfeld
2f48c65752 Explicitly create FunctionDecl in DeclExtractor
The function EnforceInitOrder() was using ImplicitlyDefineFunction()
as a shortcut to define a function returning an int. Future upgrades
of LLVM will add an assert in that function because it is not allowed
to be used when compiling C++, which Cling obviously does.
2023-01-23 18:44:07 +01:00
Vassil Vassilev
2129e96274 Do not register LLVMSupport as a library when it should be a component (#11703)
Do not register LLVMSupport as a library when it should be a component

LLVM component must be registered as LLVM_LINK_COMPONENTS to be
compatible with LLVM Dylib. Otherwise they are loaded twice in the final
binary, once through LLVM Dylmib and once through individual component,
and this results in some options being registered twice.

Fix root-project/cling#440
2023-01-20 16:59:05 +01:00
Jonas Hahnfeld
e1ae677300 Fix in-process lookup of symbols without GlobalPrefix
On Windows, C++ symbols are prefixed not with the "global" prefix,
but with a question mark.
2023-01-20 08:44:02 +01:00
Jonas Hahnfeld
10912bdab3 Remove explicit setting of COFF binary format on Windows
This is the default anyway.
2023-01-20 08:44:02 +01:00
Axel Naumann
931552f385 Silence macOS linker warnings on -undefined dynamic_lookup:
See https://github.com/root-project/root/issues/11937
for why that is currently needed. `flat_namespace` is not a solution.
2023-01-19 12:59:06 +01:00
Axel Naumann
0c9821959f Revert "[cmake] Xcode 14 warns about -undefined dynamic_lookup; simply suppress:"
See https://github.com/root-project/root/issues/11937
for why `-flat_namespace` is bad.

This reverts commit a05d4beded1481a86a65e450826352eb528b03e7.
2023-01-19 12:59:06 +01:00
Axel Naumann
feb0db382e For MSVC weak symbols, re-use even small funcs:
Without, the JIT fails to compile roottest/root/aclic/misc/assertROOT7027.C
correctly.
2023-01-17 16:59:06 +01:00
Axel Naumann
8a87d1f8d6 JIT: find existing weak symbol without materializer:
m_JIT.getSymbolAddress() invokes the symbol materializers, which compile (which
is sort of okay) but also try autoloading (which totally is not okay).

Instead, implement a function to search existing JIT symbols.

This can be accelerated by looking up the whole set of symbols, instead of doing it
symbol by symbol. I leave that refactoring for later...
2023-01-17 16:59:06 +01:00
Axel Naumann
1b6158779e JIT: separate re-use of weak sym vars, funcs (NFCI). 2023-01-17 16:59:06 +01:00
Axel Naumann
2925ee3a32 JIT: only re-use *large* existing weak functions:
Small functions might get inlined, and hiding their definition prevents
the inliner from doing its job.
2023-01-17 16:59:06 +01:00
Axel Naumann
4b2e5f8e2c Reuse existing weak symbols, also from JIT:
With the upgrade to llvm-13, the JIT lost the ability to re-use existing weak
symbols that the JIT had already emitted, instead only looking at dlsym. This
causes a significant increase in JITted symbols, and thus a significant slow-down
of cling / its JIT.

This restores the old behavior, with an identical set of symbols that jet jitted.
2023-01-17 16:59:06 +01:00
Jonas Hahnfeld
51e15ea523 Prepare for monorepo layout 2023-01-16 14:59:08 +01:00
Axel Naumann
da247bd77a Re-enable CodeGen-OptLevel after llvm13 upgrade:
With the upgrade, BackendPasses was modifying a TargetMachine that was
not used by SimpleCompiler.

Change that by
- using a SimpleCompiler that uses IncrementalJIT::TM;
- moving the TM creation to IncrementalJIT, and giving access to it

This reduces the runtime of https://github.com/root-project/root/issues/11927
to
- before llvm upgrade: 2.69s
- llvm13, without this commit: ???
- llvm13, with this commit: 2.89s
i.e, a slow-down of 7% (that is likely caused by the different emission
mechanism of Orc-v2; to be confirmed...)
2022-12-21 17:14:02 +01:00
Vassil Vassilev
cefa80ef54 Rebase better to llvm13 to the FileEntryRef and FileEntry.
Likely fixes root-project/root#11933
2022-12-20 08:59:05 +01:00
Guilherme Amadio
b0940d1ba7 Re-enable support for profiling/debugging interpreted/JITted code
This feature, originally added in commit 22b1606f, was reverted to
make the LLVM upgrade to version 13 easier. This commit adds back
all functionality as it was just before the LLVM upgrade.
2022-12-15 10:29:03 +01:00
Jonas Hahnfeld
b9ec7687bf Create non-empty source file
LLVM currently has an issue that a completely empty source file is
stored as a nullptr, but DIFile::getSource() does not correctly deal
with this situation. A crash can be observed if just attempting to
launch root.exe with EXTRA_CLING_ARGS="-gdwarf-5 -gembed-source".

See https://reviews.llvm.org/D137152 and https://reviews.llvm.org/D138658
for discussions with upstream.
2022-12-13 09:59:05 +01:00
Jonas Hahnfeld
092c9c981f Increase initial offset into virtual file
Otherwise the PresumedLoc's suddely get valid once we avoid a
completely empty soure buffer in the next commit.
2022-12-13 09:59:05 +01:00
Bertrand Bellenot
9cfaafc10e Fix compilation with VS 2022 v17.4 (#11866)
* Fix compilation with VS 2022 v17.4

This fixes the issue #10875 _HAS_CONDITIONAL_EXPLICIT=0 won't work with VS 2022 17.4

* In fact with LLVM 13 this workaround is not needed anymore
2022-12-12 11:29:05 +01:00
Axel Naumann
7bcc42aa20 Make string_view module non-textual:
this causes issues with C++20 module composability and the need for textual is unclear.
2022-12-11 22:14:05 +01:00
Axel Naumann
7aac81b4c0 Fix bug in handling EXTRA_CLING_ARGS:
The splitting code requires the full string to be null-terminated.
Before, random bytes were attached to the last option.
2022-12-09 08:44:20 +01:00
Axel Naumann
b0cb236016 Track (and unload) symbols from dylibs:
When creating orec-Symbols for dylib symbols, reloading the
dylib might mean a change in symbol (address). So unloading a
dylib means we need to unload the orc-Symbol.

This is implemented through resource-tracking the symbols as
provided by DynamicLibrarySearchGenerator. Actually, as
DynamicLibrarySearchGenerator does not support resource tracking,
it is implemented in a near-copy of DynamicLibrarySearchGenerator,
RTDynamicLibrarySearchGenerator, which uses the transaction of the
most recent module for the ResourceTracker.
2022-12-09 08:44:20 +01:00
Axel Naumann
8a267f84e3 Remove unused data members. 2022-12-09 08:44:20 +01:00
Axel Naumann
0950faa211 Remove spurrious string generation. 2022-12-09 08:44:20 +01:00
Jonas Hahnfeld
1f237a50ee Disable release of code and data sections
This is required until CallFunc is informed about unloading, and can
re-generate the wrapper (if the decl is still available).
2022-12-09 08:44:20 +01:00
Jonas Hahnfeld
4f3af1dc2c Reintroduce custom memory manager
This is still required for proper exception handling support. See
commits 3f74182697 and a7b0b3e647 by Philippe for details on the
problem and the original code, which I was able to significantly
simplify with the new JIT infrastructure.

For the moment, this disables JITLink even on platforms that had it
active by default (notably macOS). This will be reintroduced at a
later point, which will require a memory manager implementation for
the JITLink interface.
2022-12-09 08:44:20 +01:00
Axel Naumann
13b4be1603 Make IncrementalExecutor::replaceSymbol() take IR names:
Previously, one needed to pass linker-mangled names, which exposes details that
clients of IncrementalExecutor should not have to deal with. Instead, use the IR
name and do the linker-mangling in IncrementalJIT::addOrReplaceDefinition().

This fixes the lack of static destruction on macOS, visible e.g. in the test
failure of roottest/cling/staticinit/ROOT-7775.
2022-12-09 08:44:20 +01:00
Jonas Hahnfeld
76d96e317f Mark __dso_handle a "far" pointer:
The relocation needs to allow for long offsets, as for the JIT, __dso_handle
might be outside the shared library. Fixes
```
cling JIT session error: In graph cling-module-10-jitted-objectbuffer, section __TEXT,__StaticInit: relocation target "___dso_handle" at address 0x7fe1ee5052e0 is out of range of Delta32 fixup at 0x108c410bd (___cxx_global_var_initcling_module_10_, 0x108c41090 + 0x2d)
[runStaticInitializersOnce]: Failed to materialize symbols: { (main, { $.cling-module-10.__inits.0, __ZN12IncidentTypeL2m1E, __ZN6MarkerD2Ev, __ZN6MarkerD1Ev, ___cxx_global_var_initcling_module_10_.1, __GLOBAL__sub_I_cling_module_10, __ZN6MarkerC2EPKc, ___cxx_global_var_initcling_module_10_.3, __ZN12IncidentTypeL2m3E, __ZN6MarkerC1EPKc, __ZN12IncidentTypeL2m2E, ____orc_init_func.cling-module-10, ___cxx_global_var_initcling_module_10_ }) }
```
as seen no RISC-V and macOS, i.e. with the JITLinker.
2022-12-09 08:44:20 +01:00
Vassil Vassilev
89543f66cf Add an absolute symbol pointing to _CxxThrowException on Windows 2022-12-09 08:44:20 +01:00
Axel Naumann
21f2c66a0f Delete globals before JIT deletes the module:
Fixes root-meta-execUnloading which shows
```==768591== Invalid read of size 8
==768591==    at 0x6EFB10E: llvm::ilist_node_base<false>::getNext() const (ilist_node_base.h:29)
==768591==    by 0x6FBF119: llvm::ilist_node_impl<llvm::ilist_detail::node_options<llvm::Function, false, false, void> >::getNext() (ilist_node.h:66)
==768591==    by 0x6FBED48: llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Function, false, false, void>, false, false>::operator++() (ilist_iterator.h:157)
==768591==    by 0x6FBEA20: llvm::simple_ilist<llvm::Function>::begin() (simple_ilist.h:117)
==768591==    by 0x6FBE40F: llvm::Module::begin() (Module.h:622)
==768591==    by 0x6FBE475: llvm::Module::functions() (Module.h:634)
==768591==    by 0x6FBE22F: cling::TransactionUnloader::unloadModule(llvm::Module*) (TransactionUnloader.cpp:175)
==768591==    by 0x6FBDF83: cling::TransactionUnloader::RevertTransaction(cling::Transaction*) (TransactionUnloader.cpp:126)
==768591==    by 0x6F90CD0: cling::Interpreter::unload(cling::Transaction&) (Interpreter.cpp:1553)
==768591==    by 0x6F90E25: cling::Interpreter::unload(unsigned int) (Interpreter.cpp:1574)
==768591==    by 0x721A9F2: cling::MetaSema::actOnUndoCommand(unsigned int) (MetaSema.cpp:186)
```
...
```
==768591==  Address 0x144b0ce0 is 32 bytes inside a block of size 800 free'd
==768591==    at 0x484399B: operator delete(void*, unsigned long) (vg_replace_malloc.c:935)
==768591==    by 0x6FBC7CA: std::default_delete<llvm::Module>::operator()(llvm::Module*) const (unique_ptr.h:95)
==768591==    by 0x6FBC82D: std::__uniq_ptr_impl<llvm::Module, std::default_delete<llvm::Module> >::reset(llvm::Module*) (unique_ptr.h:203)
==768591==    by 0x6FBC87E: std::unique_ptr<llvm::Module, std::default_delete<llvm::Module> >::reset(llvm::Module*) (unique_ptr.h:501)
==768591==    by 0x6FBC04E: std::unique_ptr<llvm::Module, std::default_delete<llvm::Module> >::operator=(decltype(nullptr)) (unique_ptr.h:434)
==768591==    by 0x70E60E7: llvm::orc::ThreadSafeModule::~ThreadSafeModule() (ThreadSafeModule.h:116)
==768591==    by 0x936EC79: llvm::orc::IRMaterializationUnit::~IRMaterializationUnit() (Layer.h:31)
==768591==    by 0x93707F5: llvm::orc::BasicIRLayerMaterializationUnit::~BasicIRLayerMaterializationUnit() (Layer.h:121)
==768591==    by 0x9370811: llvm::orc::BasicIRLayerMaterializationUnit::~BasicIRLayerMaterializationUnit() (Layer.h:121)
==768591==    by 0x6E91B45: std::default_delete<llvm::orc::MaterializationUnit>::operator()(llvm::orc::MaterializationUnit*) const (unique_ptr.h:95)
==768591==    by 0x6E8F9A3: std::unique_ptr<llvm::orc::MaterializationUnit, std::default_delete<llvm::orc::MaterializationUnit> >::~unique_ptr() (unique_ptr.h:396)
==768591==    by 0x935D3C3: llvm::orc::JITDylib::UnmaterializedInfo::~UnmaterializedInfo() (Core.h:1062)
==768591==    by 0x935D3DE: void std::_Destroy<llvm::orc::JITDylib::UnmaterializedInfo>(llvm::orc::JITDylib::UnmaterializedInfo*) (stl_construct.h:151)
==768591==    by 0x935D26B: void std::allocator_traits<std::allocator<void> >::destroy<llvm::orc::JITDylib::UnmaterializedInfo>(std::allocator<void>&, llvm::orc::JITDylib::UnmaterializedInfo*) (alloc_traits.h:648)
==768591==    by 0x935CBA0: std::_Sp_counted_ptr_inplace<llvm::orc::JITDylib::UnmaterializedInfo, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() (shared_ptr_base.h:613)
==768591==    by 0x6C39FCE: std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() (shared_ptr_base.h:346)
==768591==    by 0x6C3CB36: std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count() (shared_ptr_base.h:1071)
==768591==    by 0x6E98CED: std::__shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr() (shared_ptr_base.h:1524)
==768591==    by 0x6E98D09: std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo>::~shared_ptr() (shared_ptr.h:175)
==768591==    by 0x932D614: llvm::DenseMapBase<llvm::DenseMap<llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo>, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo> > >, llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo>, llvm::DenseMapInfo<llvm::orc::SymbolStringPtr>, llvm::detail::DenseMapPair<llvm::orc::SymbolStringPtr, std::shared_ptr<llvm::orc::JITDylib::UnmaterializedInfo> > >::erase(llvm::orc::SymbolStringPtr const&) (DenseMap.h:307)
==768591==    by 0x931028D: llvm::orc::JITDylib::removeTracker(llvm::orc::ResourceTracker&) (Core.cpp:1510)
==768591==    by 0x9315060: llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&)::{lambda()#1}::operator()() const (Core.cpp:2155)
==768591==    by 0x93150D0: decltype(auto) llvm::orc::ExecutionSession::runSessionLocked<llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&)::{lambda()#1}>(llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&)::{lambda()#1}&&) (Core.h:1326)
==768591==    by 0x9315272: llvm::orc::ExecutionSession::removeResourceTracker(llvm::orc::ResourceTracker&) (Core.cpp:2152)
==768591==    by 0x93060FD: llvm::orc::ResourceTracker::remove() (Core.cpp:53)
==768591==    by 0x70F1D58: cling::IncrementalJIT::removeModule(cling::Transaction const&) (IncrementalJIT.cpp:151)
==768591==    by 0x6FBE6CF: cling::IncrementalExecutor::unloadModule(cling::Transaction const&) const (IncrementalExecutor.h:180)
==768591==    by 0x6FBDE5B: cling::TransactionUnloader::RevertTransaction(cling::Transaction*) (TransactionUnloader.cpp:119)
==768591==    by 0x6F90CD0: cling::Interpreter::unload(cling::Transaction&) (Interpreter.cpp:1553)
==768591==    by 0x6F90E25: cling::Interpreter::unload(unsigned int) (Interpreter.cpp:1574)
==768591==    by 0x721A9F2: cling::MetaSema::actOnUndoCommand(unsigned int) (MetaSema.cpp:186)
```
2022-12-09 08:44:20 +01:00
Axel Naumann
3675713fee Remove unused data members, parameters, includes.
(cherry picked from commit 872d7c14d8c8e2883ed20a0b6c66a4bbafa4237d)
2022-12-09 08:44:20 +01:00
Axel Naumann
a319dbff2d Formatting (80cols). 2022-12-09 08:44:20 +01:00
Axel Naumann
d8c5c2a206 Pass libCling handle at construction time, resolve from it:
OrcV2 uses specific DyLibs to resolve symbols from, we need a dedicated
resolver for libCling because its symbols cannot be found from the
process.

Remove now unused "ExposeHiddenSharedLibrarySymbols".
2022-12-09 08:44:20 +01:00
Axel Naumann
0362d3fcda Make in-process sym lookup configurable. 2022-12-09 08:44:20 +01:00
Axel Naumann
7347463577 weak symbol re-emission veto: only consider in-process symbols:
Asking for JIT symbols triggers autoloading. JIT symbols will be
resolved by the JIT itself.
2022-12-09 08:44:20 +01:00
Vassil Vassilev
c11799bb99 Explicitly instantiate pending instantiations.
See llvm/llvm-project@4f3e3811d1
2022-12-09 08:44:20 +01:00
Simeon Ehrig
0ee3ebcb5b Fixed bug in symbol linking for CUDA kernel registration with LLVM 13.
Since cling was ported to LLVM 13, it is required that the
`__cuda_register_globals` function and the `__cuda_fatbin_wrapper` and
`__cuda_gpubin_handle` variables are unique when defining a CUDA kernel.
Otherwise, the JIT is lazy and reuses the compiled version of
`__cuda_register_globals`, `__cuda_fatbin_wrapper` and `__cuda_gpubin_handle`
from the first CUDA kernel definition for all subsequent CUDA kernel
definitions, which in practice means that the PTX code from the first kernel is
re-registered each time.

Increase the default CUDA SM level to 35 because SM 20 is deprecated or
removed in the current CUDA SDK versions.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
927f761b8b TCling: Re-implement autoload via MaterializationUnits
Replace the existing LazyFunctionCreator interface by a function
to add DefinitionGenerators to be passed via the chain Interpreter
-> IncrementalExecutor -> IncrementalJIT.
Implement a DefinitionGenerator in TCling(Callbacks) to define
MaterializationUnits for autoloading symbols. This also allows to
remove the double DynamicLibrarySearchGenerator now that the created
AutoloadLibraryMUs inject the addresses into the JIT after loading
the required library.
2022-12-09 08:44:19 +01:00
Axel Naumann
619f5e83a2 Silence Orc missing symbols message:
IncrementalExecutor has its own, showing what is requesting the
symbols, which is more useful than the diagnostic of Orc. OTOH
Orc might emit more errors than just missing symbols, and if
there are multiple missing symbols, IncrementalExecutor only
shows the first.

So enable `Verbose` also for IncrementalJIT, showing the original
missing symbols as diagnosed by Orc.

This fixes roottest_cling_other_checkMissingSymbolExitCode which
fails due to the extra diagnostic error line.
2022-12-09 08:44:19 +01:00
Axel Naumann
c28fc71858 Remove duplicate / impossible condition. 2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
f63fdda04e Weaken definitions of typeinfos and vtables
Introduce a backend pass that changes the linkage of external symbols
starting with "_ZT" (ie. typeinfo names, typeinfos, and vtables) to
weak symbols. This avoids duplicate symbol errors with the new JIT
linker.

FIXME: This is a hack, we should teach the frontend to emit these
only once, or mark all duplicates as available_externally (if that
improves performance due to optimizations).
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
e338773863 Avoid duplicate symbols from RuntimeUniverse.h
Make InterpreterGeneratedCodeDiagnosticsMaybeIncorrect static, which
is then turned into a weak symbol by KeepLocalGVPass.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
d8451f6822 Print default template arguments
Fixes (at least) roottest-cling-stl-default-VectorSort and
roottest-cling-templateMembers-runoperatorEqual.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
3bbcfe34b1 Ignore duplicate kept GlobalValues
Set WeakAnyLinkage instead of ExternalLinkage to tell the JIT linker
that it should not error on duplicate symbols.
2022-12-09 08:44:19 +01:00
Jonas Hahnfeld
b721467e15 Fix emission of vtables
It is (again) necessary to call DefineUsedVTables() to enable (at
least) the emission of implicitly defined destructors overriding
the virtual destructor in a base class, see the added test.
2022-12-09 08:44:19 +01:00
Vassil Vassilev
ed72130ee6 Fix the special case for Windows. FIXE: Try remove ifdef 2022-12-09 08:44:19 +01:00
Vassil Vassilev
b4b96a6d85 The JIT now is more consistent on OSX with the linker-level mangling
OSX (due to legacy to disambigate between asm and C symbols) implements an extra
linker-level mangling which adds another `_` in the name. However, that is only
in the on-disk representation and needs to be dropped upon calling dlsym.

OrcV2 gives us a handle to the symbol with starts with `__` allowing us to
invert the logic of the symbol search where we drop the `__` for dlsym.
2022-12-09 08:44:19 +01:00