1871 Commits

Author SHA1 Message Date
Michal Simek
c590974629 ARM64: zynqmp: Add eeprom memories on i2c bus
Add i2c eeprom memories on i2c bus.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
Reviewed-by: Shubhrajyoti Datta <shubhraj@xilinx.com>
2015-07-31 10:46:29 +02:00
Michal Simek
34ad39b145 ARM64: zynqmp: Enable sdhci on ep108
Enable both sdhcis on ep108.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:29 +02:00
Michal Simek
c7c09d192f ARM64: zynqmp: Enable watchdog on ep108
Enable watchdog on ep108.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:28 +02:00
Michal Simek
22eda14afc ARM64: zynqmp: Add DWC3 usb support
Add usb nodes to DTSI and enable both of them on ep108.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:27 +02:00
Michal Simek
ff92e3614a ARM64: zynqmp: Add SMMU support
Add SMMU DT node to DTSI.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:27 +02:00
Michal Simek
3a8691f530 ARM64: zynqmp: Add CANs node for platform
Also enable can0 for ep108.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:26 +02:00
Michal Simek
b72b44b617 ARM64: zynqmp: Use zynqmp specific compatible string for gpio
The patch:
"gpio: Added support to Zynq Ultrascale+ MPSoC"
(sha1: bdf7a4ae371894b4dc10b5820006b0a82d484929)
added zynqmp specific features. This patch is switching the driver to
use the zynqmp compatible string.
Also enable the driver for ep108 platform.

Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:25 +02:00
Suneel Garapati
8fae442f88 devicetree: xilinx: zynqmp: add sata node
add sata node with sata fixed clock nodes in dtsi file.
enable sata in zynqmp-ep108.dts with broken-gen2.

Signed-off-by: Suneel Garapati <suneel.garapati@xilinx.com>
Signed-off-by: Michal Simek <michal.simek@xilinx.com>
2015-07-31 10:46:25 +02:00
Lorenzo Pieralisi
72407514c9 ARM64: PCI: do not enable resources on PROBE_ONLY systems
On ARM64 PROBE_ONLY PCI systems resources are not currently claimed,
therefore they can't be enabled since they do not have a valid
parent pointer; this in turn prevents enabling PCI devices on
ARM64 PROBE_ONLY systems, causing PCI devices initialization to
fail.

To solve this issue, resources must be claimed when devices are
added on PROBE_ONLY systems, which ensures that the resource hierarchy
is validated and the resource tree is sane, but this requires changes
in the ARM64 resource management that can affect adversely existing
PCI set-ups (claiming resources on !PROBE_ONLY systems might break
existing ARM64 PCI platform implementations).

As a temporary solution in preparation for a proper resources claiming
implementation in ARM64 core, to enable PCI PROBE_ONLY systems on ARM64,
this patch adds a pcibios_enable_device() arch implementation that
simply prevents enabling resources on PROBE_ONLY systems (mirroring ARM
behaviour).

This is always a safe thing to do because on PROBE_ONLY systems the
configuration space set-up can be considered immutable, and it is in
preparation of proper resource claiming that would finally validate
the PCI resources tree in the ARM64 arch implementation on PROBE_ONLY
systems.

For !PROBE_ONLY systems resources enablement in pcibios_enable_device()
on ARM64 is implemented as in current PCI core, leaving the behaviour
unchanged.

Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-30 20:17:07 +01:00
Will Deacon
a14949e09a arm64: cmpxchg: truncate sub-word signed types before comparison
When performing a cmpxchg operation on a signed sub-word type (e.g. s8),
we need to ensure that the upper register bits of the "old" value used
for comparison are zeroed, otherwise we may erroneously fail the cmpxchg
which may even be interpreted as success by the caller (if the compiler
performs the truncation as part of its check). This has been observed
in mod_state, where negative values where causing problems with
this_cpu_cmpxchg.

This patch fixes the issue by explicitly casting 8-bit and 16-bit "old"
values using unsigned types in our cmpxchg wrappers. 32-bit types can be
left alone, since the underlying asm makes use of W registers in this
case.

Reported-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-30 20:16:53 +01:00
Will Deacon
ef5e724b25 arm64: alternative: put secondary CPUs into polling loop during patch
When patching the kernel text with alternatives, we may end up patching
parts of the stop_machine state machine (e.g. atomic_dec_and_test in
ack_state) and consequently corrupt the instruction stream of any
secondary CPUs.

This patch passes the cpu_online_mask to stop_machine, forcing all of
the CPUs into our own callback which can place the secondary cores into
a dumb (but safe!) polling loop whilst the patching is carried out.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-30 19:07:28 +01:00
Ray Jui
6aad8bf993 arm64: dts: Add Broadcom North Star 2 support
Add Broadcom NS2 device tree binding document. Also add initial device
tree dtsi for Broadcom North Star 2 (NS2) SoC and board support for NS2
SVK board

Signed-off-by: Jon Mason <jonmason@broadcom.com>
Signed-off-by: Ray Jui <rjui@broadcom.com>
Reviewed-by: Scott Branden <sbranden@broadcom.com>
Signed-off-by: Olof Johansson <olof@lixom.net>
2015-07-29 22:07:11 +02:00
Ray Jui
36b7c5831d arm64: Add Broadcom iProc family support
This patch adds support to Broadcom's iProc family of arm64 based SoCs
in the arm64 Kconfig and defconfig files

Signed-off-by: Ray Jui <rjui@broadcom.com>
Reviewed-by: Scott Branden <sbranden@broadcom.com>
Signed-off-by: Olof Johansson <olof@lixom.net>
2015-07-29 22:07:08 +02:00
Will Deacon
484c96dbb2 arm64: lse: fix lse cmpxchg code indentation
For some reason, the ll/sc cmpxchg asm is all off to the left and
awkward to read in conjunction with the following (correctly indented)
LSE version.

This patch shifts the ll/sc code back to where it should be.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-29 18:32:09 +01:00
Jonas Rabenstein
63a581865e arm64: remove redundant object file list
Commit 4b3dc9679cf7 ("arm64: force CONFIG_SMP=y and remove redundant
#ifdefs") forces SMP on arm64. To build the necessary objects for SMP,
they were added to the arm64-obj-y rule in arch/arm64/kernel/Makefile,
without removing the arm64-obj-$(CONFIG_SMP) rule.

Remove redundant object file list depending on always-yes CONFIG_SMP in
arch/arm64/kernel/Makefile.

Signed-off-by: Jonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-29 18:32:09 +01:00
Jonas Rabenstein
377bcff9a3 arm64: remove dead-code depending on CONFIG_UP_LATE_INIT
Commit 4b3dc9679cf7 ("arm64: force CONFIG_SMP=y and remove redundant
and therfore can not be selected anymore.

Remove dead #ifdef-block depending on UP_LATE_INIT in
arch/arm64/kernel/setup.c

Signed-off-by: Jonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
[will: kill do_post_cpus_up_work altogether]
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-29 18:32:09 +01:00
Ivan T. Ivanov
9943230c88 arm64: dts: qcom: Add apq8016-sbc board LED's related device nodes
APQ8016 SBC board have 6 user controllable LED's.

Add following devices:

LED1 green LED triggered by system heartbeat.
LED2 green LED triggered by access to eMMC device.
LED3 green LED triggered by access to SD card.
LED4 green LED no trigger assigned.
LED5 yellow LED triggered by access to WLAN.
LED6 blue LED triggered by access to Bluetooth.

Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:13 -05:00
Ivan T. Ivanov
8883ad80c1 arm64: dts: qcom: Fix apq8016-sbc board USB related pin definitions
USB2513B HUB reset line is connected to PMIC GPIO3 not GPIO1.

Fix TC7USB40MU Dual SPDT Switch select input line control, which is
connected to PMIC GPIO4 not GPIO2 and disable the pin. It is not used
for now.

Remove user LEDs definitions, because they clash with above numbers.

Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:13 -05:00
Ivan T. Ivanov
c8077068f7 arm64: dts: qcom: apq8016-sbc: Don't hog client driver pins
Hogging pins from pinctrl driver prevents client drivers
to probe.

Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:13 -05:00
Ivan T. Ivanov
596008653f arm64: dts: qcom: Add msm8916 USB configuration nodes
Add Host, Device and OTG configuration nodes.

Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:13 -05:00
Srinivas Kandagatla
c4da5a5616 arm64: dts: qcom: Add msm8916 sdhci configuration nodes
Add sdhci1 and sdhci2 device configuration nodes.

Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:13 -05:00
Ivan T. Ivanov
a0e5fb1031 arm64: dts: qcom: Add msm8916 BLSP device nodes
Add device nodes for SPI1, SPI2, SPI3, I2C4, SPI5, SPI6 and
BAM(DMA) engine connected to them.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:12 -05:00
Ivan T. Ivanov
1b08a582a0 arm64: dts: qcom: Extend msm8916 pinctrl device coverage
Create separate file for MSM8916 pinctrl default/sleep pins state
definitions. Move in UART2 states and add SPI, I2C and SDC configurations.

Signed-off-by: Stanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: Ivan T. Ivanov <ivan.ivanov@linaro.org>
Signed-off-by: Andy Gross <agross@codeaurora.org>
2015-07-28 16:19:12 -05:00
Olof Johansson
af429d9ebf - arm64: dts: mediatek: add mt6795 support
- Document: DT: Add bindings for mediatek MT6795 SoC Platform
 - arm64: dts: mediatek: Add MT8173 MMC dts
 - arm64: dts: mt8173: Add afe device node
 - arm64: dts: mt8173-evb: Add PMIC support
 - dts: mt8173-evb: Add da9211 to i2c1
 - ARM: dts: mt8173: support arm64 cpuidle-dt
 - ARM64: MediaTek MT8173: Add SCPSYS device node
 - arm64: dts: mt8173: Add I2C device node
 - arm64: dts: mt8173: Add watchdog device node
 - arm64: dts: mt8173: Add PMIC wrapper device node
 - arm64: dts: mt8173: Use real clock for UARTs
 - arm64: dts: mt8173: Add clock controller device nodes
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJVt2JqAAoJELQ5Ylss8dNDh28P/i+8RDwaK85GgJtC3gYmjTPU
 JwpNYbzyCD6ObhPQS7arGrn+guyVSrQid95V1YNVQJRufuHGJzRWQ7v0i8p3d144
 b3eLOWRF2Tuvx6fDy2sGHiFuWvprtpYvhiM12A8nVllUcV1HpY8nndE3Cffrx8JX
 RY2o/x62beSzzOnsXny/ye5fqVitMfoNC//b1FB5qnEs8wl+SGW3PVGBPeOsHp6T
 U+sj4A31lbKtUL5JWsaHIM/cgNrkh3NcuU1cm9p5dBGIqh2ZsuiyHyjrBoERYrbw
 nNwGlwRPIq/XosnQIr+pjlsJ4yJrYEavord8ve58N4fBD6tmeQFExEP5g1T797px
 7tL5Mw7FT95J7s1eYqtV8ZUTdvMZvLGNPGX7sQsPIezpCra9h9i0XaSkCbrsikKY
 DF0C1vtF86XS+Ki7Oxll5z3PYd1Hv8NUDl/h5o5nnFOXZz6RE2PT6mM+YUPLeA/B
 An7+GlWYe6lmcprPMNYkhf1ry0srAJKMBi1xAV2DkZKho1qNyvVQIJVEVm7m6yXp
 DqD8+gBKnbeNOs+s6h/JHDqcHVtoGgnWByMGcUcb86LJwzjKz321GEUBJJ6HMobu
 vkY379//teNbhJQz3E5BcAiYMKszc1Le8cwLGD+G5tI3toR5GRagJrQV0evS/Py4
 q2sLamn1vn03aUEvskkT
 =RiLC
 -----END PGP SIGNATURE-----

Merge tag 'v4.2-next-arm64' of https://github.com/mbgg/linux-mediatek into next/arm64

- arm64: dts: mediatek: add mt6795 support
- Document: DT: Add bindings for mediatek MT6795 SoC Platform
- arm64: dts: mediatek: Add MT8173 MMC dts
- arm64: dts: mt8173: Add afe device node
- arm64: dts: mt8173-evb: Add PMIC support
- dts: mt8173-evb: Add da9211 to i2c1
- ARM: dts: mt8173: support arm64 cpuidle-dt
- ARM64: MediaTek MT8173: Add SCPSYS device node
- arm64: dts: mt8173: Add I2C device node
- arm64: dts: mt8173: Add watchdog device node
- arm64: dts: mt8173: Add PMIC wrapper device node
- arm64: dts: mt8173: Use real clock for UARTs
- arm64: dts: mt8173: Add clock controller device nodes

* tag 'v4.2-next-arm64' of https://github.com/mbgg/linux-mediatek:
  arm64: dts: mediatek: add mt6795 support
  Document: DT: Add bindings for mediatek MT6795 SoC Platform
  arm64: dts: mediatek: Add MT8173 MMC dts
  arm64: dts: mt8173: Add afe device node
  arm64: dts: mt8173-evb: Add PMIC support
  dts: mt8173-evb: Add da9211 to i2c1
  ARM: dts: mt8173: support arm64 cpuidle-dt
  ARM64: MediaTek MT8173: Add SCPSYS device node
  arm64: dts: mt8173: Add I2C device node
  arm64: dts: mt8173: Add watchdog device node
  arm64: dts: mt8173: Add PMIC wrapper device node
  arm64: dts: mt8173: Use real clock for UARTs
  arm64: dts: mt8173: Add clock controller device nodes

Signed-off-by: Olof Johansson <olof@lixom.net>
2015-07-28 18:16:13 +02:00
Will Deacon
766ffb6980 arm64: pgtable: fix definition of pte_valid
pte_valid should check if the PTE_VALID bit (1 << 0) is set in the pte,
so fix the macro definition to use bitwise & instead of logical &&.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-28 16:14:03 +01:00
Will Deacon
c1d7cd228b arm64: spinlock: fix ll/sc unlock on big-endian systems
When unlocking a spinlock, we perform a read-modify-write on the owner
ticket in order to increment it and store it back with release
semantics.

In the LL/SC case, we load the 16-bit ticket using a 32-bit load and
therefore store back the wrong halfword on a big-endian system,
corrupting the lock after the first unlock and killing the system dead.

This patch fixes the unlock code to use 16-bit accessors consistently.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-28 14:48:00 +01:00
Catalin Marinas
4150e50bf5 arm64: Use last level TLBI for user pte changes
The flush_tlb_page() function is used on user address ranges when PTEs
(or PMDs/PUDs for huge pages) were changed (attributes or clearing). For
such cases, it is more efficient to invalidate only the last level of
the TLB with the "tlbi vale1is" instruction.

In the TLB shoot-down case, the TLB caching of the intermediate page
table levels (pmd, pud, pgd) is handled by __flush_tlb_pgtable() via the
__(pte|pmd|pud)_free_tlb() functions and it is not deferred to
tlb_finish_mmu() (as of commit 285994a62c80 - "arm64: Invalidate the TLB
corresponding to intermediate page table levels"). The tlb_flush()
function only needs to invalidate the TLB for the last level of page
tables; the __flush_tlb_range() function gains a fourth argument for
last level TLBI.

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-28 11:44:01 +01:00
Catalin Marinas
da4e73303e arm64: Clean up __flush_tlb(_kernel)_range functions
This patch moves the MAX_TLB_RANGE check into the
flush_tlb(_kernel)_range functions directly to avoid the
undescore-prefixed definitions (and for consistency with a subsequent
patch).

Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-28 11:43:15 +01:00
Mark Rutland
c53e0baa6f arm64: mm: mark create_mapping as __init
Currently create_mapping is marked with __ref, apparently because it
refers to early_alloc. However, create_mapping has no logic to prevent
erroneous use of early_alloc after it has been freed, and is only ever
called by __init functions anyway. Thus the __ref marker is misleading
and unnecessary.

Instead, this patch marks create_mapping as __init, resulting in
warnings if it is used from a a non __init functions, and allowing its
memory to be reclaimed.

Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-28 11:36:09 +01:00
Will Deacon
6f883d10a1 arm64: debug: rename enum debug_el to avoid symbol collision
lib/list_sort.c defines a 'struct debug_el', where "el" is assumedly a
a contraction of "element". This conflicts with 'enum debug_el' in our
asm/debug-monitors.h header file, where "el" stands for Exception Level.

The result is build failure when targetting allmodconfig, so rename our
enum to 'dbg_active_el' to be slightly more explicit about what it is.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 18:36:54 +01:00
Wang Long
662ba3dbce arm64: mm: add __init section marker to free_initrd_mem
It is not needed after booting, this patch moves the
free_initrd_mem() function to the __init section.

This patch also make keep_initrd __initdata, to reduce kernel
size.

Signed-off-by: Wang Long <long.wanglong@huawei.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 18:29:18 +01:00
Ard Biesheuvel
f91b1feada arm64/efi: map the entire UEFI vendor string before reading it
At boot, the UTF-16 UEFI vendor string is copied from the system
table into a char array with a size of 100 bytes. However, this
size of 100 bytes is also used for memremapping() the source,
which may not be sufficient if the vendor string exceeds 50
UTF-16 characters, and the placement of the vendor string inside
a 4 KB page happens to leave the end unmapped.

So use the correct '100 * sizeof(efi_char16_t)' for the size of
the mapping.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Fixes: f84d02755f5a ("arm64: add EFI runtime services")
Cc: <stable@vger.kernel.org> # 3.16+
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
2015-07-27 18:00:05 +01:00
Will Deacon
309585b0b9 arm64: elf: use cpuid_feature_extract_field for hwcap detection
cpuid_feature_extract_field takes care of the fiddly ID register
field sign-extension, so use that instead of rolling our own version.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 16:56:17 +01:00
Will Deacon
2e94da1379 arm64: lse: use generic cpufeature detection for LSE atomics
Rework the cpufeature detection to support ISAR0 and use that for
detecting the presence of LSE atomics.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 16:37:14 +01:00
Will Deacon
0e4a07092f arm64: kconfig: group the v8.1 features together
ARMv8 CPUs do not support any of the v8.1 features, so group them
together in Kconfig to make it clear that they're part of 8.1 and not
relevant to older cores.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:54:13 +01:00
Will Deacon
c739dc83a0 arm64: lse: rename ARM64_CPU_FEAT_LSE_ATOMICS for consistency
Other CPU features follow an 'ARM64_HAS_*' naming scheme, so do the same
for the LSE atomics.

Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:54 +01:00
Will Deacon
95eff6b27c arm64: kconfig: select HAVE_CMPXCHG_LOCAL
We implement an optimised cmpxchg_local macro, so let the kernel know.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:54 +01:00
Will Deacon
db26217e6f arm64: atomic64_dec_if_positive: fix incorrect branch condition
If we attempt to atomic64_dec_if_positive on INT_MIN, we will underflow
and incorrectly decide that the original parameter was positive.

This patches fixes the broken condition code so that we handle this
corner case correctly.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:54 +01:00
Will Deacon
6059a7b6e8 arm64: atomics: implement atomic{,64}_cmpxchg using cmpxchg
We don't need duplicate cmpxchg implementations, so use cmpxchg to
implement atomic{,64}_cmpxchg, like we do for xchg already.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:53 +01:00
Will Deacon
0ea366f5e1 arm64: atomics: prefetch the destination word for write prior to stxr
The cost of changing a cacheline from shared to exclusive state can be
significant, especially when this is triggered by an exclusive store,
since it may result in having to retry the transaction.

This patch makes use of prfm to prefetch cachelines for write prior to
ldxr/stxr loops when using the ll/sc atomic routines.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:53 +01:00
Will Deacon
a82e62382f arm64: atomics: tidy up common atomic{,64}_* macros
The common (i.e. identical for ll/sc and lse) atomic macros in atomic.h
are needlessley different for atomic_t and atomic64_t.

This patch tidies up the definitions to make them consistent across the
two atomic types and factors out common code such as the add_unless
implementation based on cmpxchg.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:53 +01:00
Will Deacon
4e39715f4b arm64: cmpxchg: avoid memory barrier on comparison failure
cmpxchg doesn't require memory barrier semantics when the value
comparison fails, so make the barrier conditional on success.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:52 +01:00
Will Deacon
0bc671d3f4 arm64: cmpxchg: avoid "cc" clobber in ll/sc routines
We can perform the cmpxchg comparison using eor and cbnz which avoids
the "cc" clobber for the ll/sc case and consequently for the LSE case
where we may have to fall-back on the ll/sc code at runtime.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:52 +01:00
Will Deacon
e9a4b79565 arm64: cmpxchg_dbl: patch in lse instructions when supported by the CPU
On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of our cmpxchg_double primitives
so that the LSE casp instruction is used instead.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:52 +01:00
Will Deacon
c342f78217 arm64: cmpxchg: patch in lse instructions when supported by the CPU
On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of our cmpxchg primitives so that
the LSE cas instruction is used instead.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:51 +01:00
Will Deacon
c8366ba0fb arm64: xchg: patch in lse instructions when supported by the CPU
On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of our xchg primitives so that
the LSE swp instruction (yes, you read right!) is used instead.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:51 +01:00
Will Deacon
084f903727 arm64: bitops: patch in lse instructions when supported by the CPU
On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of our bitops functions so that
LSE atomic instructions are used instead.

Reviewed-by: Steve Capper <steve.capper@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:51 +01:00
Will Deacon
81bb5c6420 arm64: locks: patch in lse instructions when supported by the CPU
On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of our locking functions so that
LSE atomic instructions are used for spinlocks and rwlocks.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:51 +01:00
Will Deacon
c09d6a04d1 arm64: atomics: patch in lse instructions when supported by the CPU
On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of atomic_t and atomic64_t
routines so that the call-site for the out-of-line ll/sc sequences is
patched with an LSE atomic instruction when we detect that
the CPU supports it.

If binutils is not recent enough to assemble the LSE instructions, then
the ll/sc sequences are inlined as though CONFIG_ARM64_LSE_ATOMICS=n.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:50 +01:00
Will Deacon
c0385b24af arm64: introduce CONFIG_ARM64_LSE_ATOMICS as fallback to ll/sc atomics
In order to patch in the new atomic instructions at runtime, we need to
generate wrappers around the out-of-line exclusive load/store atomics.

This patch adds a new Kconfig option, CONFIG_ARM64_LSE_ATOMICS. which
causes our atomic functions to branch to the out-of-line ll/sc
implementations. To avoid the register spill overhead of the PCS, the
out-of-line functions are compiled with specific compiler flags to
force out-of-line save/restore of any registers that are usually
caller-saved.

Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
2015-07-27 15:28:50 +01:00