Merge branch 'pci/controller/dwc'

- Extend PCI_FIND_NEXT_CAP() and PCI_FIND_NEXT_EXT_CAP() to return a
  pointer to the preceding Capability (Qiang Yu)

- Add dw_pcie_remove_capability() and dw_pcie_remove_ext_capability() to
  remove Capabilities that are advertised but not fully implemented (Qiang
  Yu)

- Remove MSI and MSI-X Capabilities for DWC controllers in platforms that
  can't support them, so we automatically fall back to INTx (Qiang Yu)

- Remove MSI-X and DPC Capabilities for Qualcomm platforms that advertise
  but don't support them (Qiang Yu)

- Remove duplicate dw_pcie_ep_hide_ext_capability() function and replace
  with dw_pcie_remove_ext_capability() (Qiang Yu)

- Add ASPM L1.1 and L1.2 Substates context to debugfs ltssm_status for
  drivers that support this (Shawn Lin)

- Skip PME_Turn_Off broadcast and L2/L3 transition during suspend if link
  is not up to avoid an unnecessary timeout (Manivannan Sadhasivam)

- Revert dw-rockchip, qcom, and DWC core changes that used link-up IRQs to
  trigger enumeration instead of waiting for link to be up because the PCI
  core doesn't allocate bus number space for hierarchies that might be
  attached (Niklas Cassel)

- Make endpoint iATU entry for MSI permanent instead of programming it
  dynamically, which is slow and racy with respect to other concurrent
  traffic, e.g., eDMA (Koichiro Den)

- Use iMSI-RX MSI target address when possible to fix endpoints using
  32-bit MSI (Shawn Lin)

- Make dw_pcie_ltssm_status_string() available and use it for logging
  errors in dw_pcie_wait_for_link() (Manivannan Sadhasivam)

- Return -ENODEV when dw_pcie_wait_for_link() finds no devices, -EIO for
  device present but inactive, -ETIMEDOUT for other failures, so callers
  can handle these cases differently (Manivannan Sadhasivam)

- Allow DWC host controller driver probe to continue if device is not found
  or found but inactive; only fail when there's an error with the link
  (Manivannan Sadhasivam)

- For controllers like NXP i.MX6QP and i.MX7D, where LTSSM registers are
  not accessible after PME_Turn_Off, simply wait 10ms instead of polling
  for L2/L3 Ready (Richard Zhu)

- Use multiple iATU entries to map large bridge windows and DMA ranges when
  necessary instead of failing (Samuel Holland)

- Rename struct dw_pcie_rp.has_msi_ctrl to .use_imsi_rx for clarity (Qiang
  Yu)

- Add EPC dynamic_inbound_mapping feature bit for Endpoint Controllers that
  can update BAR inbound address translation without requiring EPF driver
  to clear/reset the BAR first, and advertise it for DWC-based Endpoints
  (Koichiro Den)

- Add EPC subrange_mapping feature bit for Endpoint Controllers that can
  map multiple independent inbound regions in a single BAR, implement
  subrange mapping, advertise it for DWC-based Endpoints, and add Endpoint
  selftests for it (Koichiro Den)

- Allow overriding default BAR sizes for pci-epf-test (Niklas Cassel)

- Make resizable BARs work for Endpoint multi-PF configurations; previously
  it only worked for PF 0 (Aksh Garg)

- Fix Endpoint non-PF 0 support for BAR configuration, ATU mappings, and
  Address Match Mode (Aksh Garg)

- Fix issues with outbound iATU index assignment that caused iATU index to
  be out of bounds (Niklas Cassel)

- Clean up iATU index tracking to be consistent (Niklas Cassel)

- Set up iATU when ECAM is enabled; previously IO and MEM outbound windows
  weren't programmed, and ECAM-related iATU entries weren't restored after
  suspend/resume, so config accesses failed (Krishna Chaitanya Chundru)

* pci/controller/dwc:
  PCI: dwc: Fix missing iATU setup when ECAM is enabled
  PCI: dwc: Clean up iATU index usage in dw_pcie_iatu_setup()
  PCI: dwc: Fix msg_atu_index assignment
  PCI: dwc: ep: Add comment explaining controller level PTM access in multi PF setup
  PCI: dwc: ep: Add per-PF BAR and inbound ATU mapping support
  PCI: dwc: ep: Fix resizable BAR support for multi-PF configurations
  PCI: endpoint: pci-epf-test: Allow overriding default BAR sizes
  selftests: pci_endpoint: Add BAR subrange mapping test case
  misc: pci_endpoint_test: Add BAR subrange mapping test case
  PCI: endpoint: pci-epf-test: Add BAR subrange mapping test support
  Documentation: PCI: endpoint: Clarify pci_epc_set_bar() usage
  PCI: dwc: ep: Support BAR subrange inbound mapping via Address Match Mode iATU
  PCI: dwc: Advertise dynamic inbound mapping support
  PCI: endpoint: Add BAR subrange mapping support
  PCI: endpoint: Add dynamic_inbound_mapping EPC feature
  PCI: dwc: Rename dw_pcie_rp::has_msi_ctrl to dw_pcie_rp::use_imsi_rx for clarity
  PCI: dwc: Fix grammar and formatting for comment in dw_pcie_remove_ext_capability()
  PCI: dwc: Use multiple iATU windows for mapping large bridge windows and DMA ranges
  PCI: dwc: Remove duplicate dw_pcie_ep_hide_ext_capability() function
  PCI: dwc: Skip waiting for L2/L3 Ready if dw_pcie_rp::skip_l23_wait is true
  PCI: dwc: Fail dw_pcie_host_init() if dw_pcie_wait_for_link() returns -ETIMEDOUT
  PCI: dwc: Rework the error print of dw_pcie_wait_for_link()
  PCI: dwc: Rename and move ltssm_status_string() to pcie-designware.c
  PCI: dwc: Return -EIO from dw_pcie_wait_for_link() if device is not active
  PCI: dwc: Return -ENODEV from dw_pcie_wait_for_link() if device is not found
  PCI: dwc: Use cfg0_base as iMSI-RX target address to support 32-bit MSI devices
  PCI: dwc: ep: Cache MSI outbound iATU mapping
  Revert "PCI: dwc: Don't wait for link up if driver can detect Link Up event"
  Revert "PCI: qcom: Enumerate endpoints based on Link up event in 'global_irq' interrupt"
  Revert "PCI: qcom: Enable MSI interrupts together with Link up if 'Global IRQ' is supported"
  Revert "PCI: qcom: Don't wait for link if we can detect Link Up"
  Revert "PCI: dw-rockchip: Enumerate endpoints based on dll_link_up IRQ"
  Revert "PCI: dw-rockchip: Don't wait for link since we can detect Link Up"
  PCI: dwc: Skip PME_Turn_Off broadcast and L2/L3 transition during suspend if link is not up
  PCI: dw-rockchip: Change get_ltssm() to provide L1 Substates info
  PCI: dwc: Add L1 Substates context to ltssm_status of debugfs
  PCI: qcom: Remove DPC Extended Capability
  PCI: qcom: Remove MSI-X Capability for Root Ports
  PCI: dwc: Remove MSI/MSIX capability for Root Port if iMSI-RX is used as MSI controller
  PCI: dwc: Add new APIs to remove standard and extended Capability
  PCI: Add preceding capability position support in PCI_FIND_NEXT_*_CAP macros

1 files changed, 267 insertions, 2 deletions

diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index 62804120cd79..0cb7af0919dc 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -33,6 +33,8 @@
 #define COMMAND_COPY			BIT(5)
 #define COMMAND_ENABLE_DOORBELL		BIT(6)
 #define COMMAND_DISABLE_DOORBELL	BIT(7)
+#define COMMAND_BAR_SUBRANGE_SETUP	BIT(8)
+#define COMMAND_BAR_SUBRANGE_CLEAR	BIT(9)
 
 #define STATUS_READ_SUCCESS		BIT(0)
 #define STATUS_READ_FAIL		BIT(1)
@@ -48,6 +50,10 @@
 #define STATUS_DOORBELL_ENABLE_FAIL	BIT(11)
 #define STATUS_DOORBELL_DISABLE_SUCCESS BIT(12)
 #define STATUS_DOORBELL_DISABLE_FAIL	BIT(13)
+#define STATUS_BAR_SUBRANGE_SETUP_SUCCESS	BIT(14)
+#define STATUS_BAR_SUBRANGE_SETUP_FAIL		BIT(15)
+#define STATUS_BAR_SUBRANGE_CLEAR_SUCCESS	BIT(16)
+#define STATUS_BAR_SUBRANGE_CLEAR_FAIL		BIT(17)
 
 #define FLAG_USE_DMA			BIT(0)
 
@@ -57,12 +63,16 @@
 #define CAP_MSI				BIT(1)
 #define CAP_MSIX			BIT(2)
 #define CAP_INTX			BIT(3)
+#define CAP_SUBRANGE_MAPPING		BIT(4)
+
+#define PCI_EPF_TEST_BAR_SUBRANGE_NSUB	2
 
 static struct workqueue_struct *kpcitest_workqueue;
 
 struct pci_epf_test {
 	void			*reg[PCI_STD_NUM_BARS];
 	struct pci_epf		*epf;
+	struct config_group	group;
 	enum pci_barno		test_reg_bar;
 	size_t			msix_table_offset;
 	struct delayed_work	cmd_handler;
@@ -76,6 +86,7 @@ struct pci_epf_test {
 	bool			dma_private;
 	const struct pci_epc_features *epc_features;
 	struct pci_epf_bar	db_bar;
+	size_t			bar_size[PCI_STD_NUM_BARS];
 };
 
 struct pci_epf_test_reg {
@@ -102,7 +113,8 @@ static struct pci_epf_header test_header = {
 	.interrupt_pin	= PCI_INTERRUPT_INTA,
 };
 
-static size_t bar_size[] = { 512, 512, 1024, 16384, 131072, 1048576 };
+/* default BAR sizes, can be overridden by the user using configfs */
+static size_t default_bar_size[] = { 131072, 131072, 131072, 131072, 131072, 1048576 };
 
 static void pci_epf_test_dma_callback(void *param)
 {
@@ -806,6 +818,155 @@ set_status_err:
 	reg->status = cpu_to_le32(status);
 }
 
+static u8 pci_epf_test_subrange_sig_byte(enum pci_barno barno,
+					 unsigned int subno)
+{
+	return 0x50 + (barno * 8) + subno;
+}
+
+static void pci_epf_test_bar_subrange_setup(struct pci_epf_test *epf_test,
+					    struct pci_epf_test_reg *reg)
+{
+	struct pci_epf_bar_submap *submap, *old_submap;
+	struct pci_epf *epf = epf_test->epf;
+	struct pci_epc *epc = epf->epc;
+	struct pci_epf_bar *bar;
+	unsigned int nsub = PCI_EPF_TEST_BAR_SUBRANGE_NSUB, old_nsub;
+	/* reg->size carries BAR number for BAR_SUBRANGE_* commands. */
+	enum pci_barno barno = le32_to_cpu(reg->size);
+	u32 status = le32_to_cpu(reg->status);
+	unsigned int i, phys_idx;
+	size_t sub_size;
+	u8 *addr;
+	int ret;
+
+	if (barno >= PCI_STD_NUM_BARS) {
+		dev_err(&epf->dev, "Invalid barno: %d\n", barno);
+		goto err;
+	}
+
+	/* Host side should've avoided test_reg_bar, this is a safeguard. */
+	if (barno == epf_test->test_reg_bar) {
+		dev_err(&epf->dev, "test_reg_bar cannot be used for subrange test\n");
+		goto err;
+	}
+
+	if (!epf_test->epc_features->dynamic_inbound_mapping ||
+	    !epf_test->epc_features->subrange_mapping) {
+		dev_err(&epf->dev, "epc driver does not support subrange mapping\n");
+		goto err;
+	}
+
+	bar = &epf->bar[barno];
+	if (!bar->size || !bar->addr) {
+		dev_err(&epf->dev, "bar size/addr (%zu/%p) is invalid\n",
+			bar->size, bar->addr);
+		goto err;
+	}
+
+	if (bar->size % nsub) {
+		dev_err(&epf->dev, "BAR size %zu is not divisible by %u\n",
+			bar->size, nsub);
+		goto err;
+	}
+
+	sub_size = bar->size / nsub;
+
+	submap = kcalloc(nsub, sizeof(*submap), GFP_KERNEL);
+	if (!submap)
+		goto err;
+
+	for (i = 0; i < nsub; i++) {
+		/* Swap the two halves so RC can verify ordering. */
+		phys_idx = i ^ 1;
+		submap[i].phys_addr = bar->phys_addr + (phys_idx * sub_size);
+		submap[i].size = sub_size;
+	}
+
+	old_submap = bar->submap;
+	old_nsub = bar->num_submap;
+
+	bar->submap = submap;
+	bar->num_submap = nsub;
+
+	ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, bar);
+	if (ret) {
+		dev_err(&epf->dev, "pci_epc_set_bar() failed: %d\n", ret);
+		bar->submap = old_submap;
+		bar->num_submap = old_nsub;
+		kfree(submap);
+		goto err;
+	}
+	kfree(old_submap);
+
+	/*
+	 * Fill deterministic signatures into the physical regions that
+	 * each BAR subrange maps to. RC verifies these to ensure the
+	 * submap order is really applied.
+	 */
+	addr = (u8 *)bar->addr;
+	for (i = 0; i < nsub; i++) {
+		phys_idx = i ^ 1;
+		memset(addr + (phys_idx * sub_size),
+		       pci_epf_test_subrange_sig_byte(barno, i),
+		       sub_size);
+	}
+
+	status |= STATUS_BAR_SUBRANGE_SETUP_SUCCESS;
+	reg->status = cpu_to_le32(status);
+	return;
+
+err:
+	status |= STATUS_BAR_SUBRANGE_SETUP_FAIL;
+	reg->status = cpu_to_le32(status);
+}
+
+static void pci_epf_test_bar_subrange_clear(struct pci_epf_test *epf_test,
+					    struct pci_epf_test_reg *reg)
+{
+	struct pci_epf *epf = epf_test->epf;
+	struct pci_epf_bar_submap *submap;
+	struct pci_epc *epc = epf->epc;
+	/* reg->size carries BAR number for BAR_SUBRANGE_* commands. */
+	enum pci_barno barno = le32_to_cpu(reg->size);
+	u32 status = le32_to_cpu(reg->status);
+	struct pci_epf_bar *bar;
+	unsigned int nsub;
+	int ret;
+
+	if (barno >= PCI_STD_NUM_BARS) {
+		dev_err(&epf->dev, "Invalid barno: %d\n", barno);
+		goto err;
+	}
+
+	bar = &epf->bar[barno];
+	submap = bar->submap;
+	nsub = bar->num_submap;
+
+	if (!submap || !nsub)
+		goto err;
+
+	bar->submap = NULL;
+	bar->num_submap = 0;
+
+	ret = pci_epc_set_bar(epc, epf->func_no, epf->vfunc_no, bar);
+	if (ret) {
+		bar->submap = submap;
+		bar->num_submap = nsub;
+		dev_err(&epf->dev, "pci_epc_set_bar() failed: %d\n", ret);
+		goto err;
+	}
+	kfree(submap);
+
+	status |= STATUS_BAR_SUBRANGE_CLEAR_SUCCESS;
+	reg->status = cpu_to_le32(status);
+	return;
+
+err:
+	status |= STATUS_BAR_SUBRANGE_CLEAR_FAIL;
+	reg->status = cpu_to_le32(status);
+}
+
 static void pci_epf_test_cmd_handler(struct work_struct *work)
 {
 	u32 command;
@@ -861,6 +1022,14 @@ static void pci_epf_test_cmd_handler(struct work_struct *work)
 		pci_epf_test_disable_doorbell(epf_test, reg);
 		pci_epf_test_raise_irq(epf_test, reg);
 		break;
+	case COMMAND_BAR_SUBRANGE_SETUP:
+		pci_epf_test_bar_subrange_setup(epf_test, reg);
+		pci_epf_test_raise_irq(epf_test, reg);
+		break;
+	case COMMAND_BAR_SUBRANGE_CLEAR:
+		pci_epf_test_bar_subrange_clear(epf_test, reg);
+		pci_epf_test_raise_irq(epf_test, reg);
+		break;
 	default:
 		dev_err(dev, "Invalid command 0x%x\n", command);
 		break;
@@ -933,6 +1102,10 @@ static void pci_epf_test_set_capabilities(struct pci_epf *epf)
 	if (epf_test->epc_features->intx_capable)
 		caps |= CAP_INTX;
 
+	if (epf_test->epc_features->dynamic_inbound_mapping &&
+	    epf_test->epc_features->subrange_mapping)
+		caps |= CAP_SUBRANGE_MAPPING;
+
 	reg->caps = cpu_to_le32(caps);
 }
 
@@ -1070,7 +1243,7 @@ static int pci_epf_test_alloc_space(struct pci_epf *epf)
 		if (epc_features->bar[bar].type == BAR_FIXED)
 			test_reg_size = epc_features->bar[bar].fixed_size;
 		else
-			test_reg_size = bar_size[bar];
+			test_reg_size = epf_test->bar_size[bar];
 
 		base = pci_epf_alloc_space(epf, test_reg_size, bar,
 					   epc_features, PRIMARY_INTERFACE);
@@ -1142,6 +1315,94 @@ static void pci_epf_test_unbind(struct pci_epf *epf)
 	pci_epf_test_free_space(epf);
 }
 
+#define PCI_EPF_TEST_BAR_SIZE_R(_name, _id)				\
+static ssize_t pci_epf_test_##_name##_show(struct config_item *item,	\
+					   char *page)			\
+{									\
+	struct config_group *group = to_config_group(item);		\
+	struct pci_epf_test *epf_test =					\
+		container_of(group, struct pci_epf_test, group);	\
+									\
+	return sysfs_emit(page, "%zu\n", epf_test->bar_size[_id]);	\
+}
+
+#define PCI_EPF_TEST_BAR_SIZE_W(_name, _id)				\
+static ssize_t pci_epf_test_##_name##_store(struct config_item *item,	\
+					    const char *page,		\
+					    size_t len)			\
+{									\
+	struct config_group *group = to_config_group(item);		\
+	struct pci_epf_test *epf_test =					\
+		container_of(group, struct pci_epf_test, group);	\
+	int val, ret;							\
+									\
+	/*								\
+	 * BAR sizes can only be modified before binding to an EPC,	\
+	 * because pci_epf_test_alloc_space() is called in .bind().	\
+	 */								\
+	if (epf_test->epf->epc)						\
+		return -EOPNOTSUPP;					\
+									\
+	ret = kstrtouint(page, 0, &val);				\
+	if (ret)							\
+		return ret;						\
+									\
+	if (!is_power_of_2(val))					\
+		return -EINVAL;						\
+									\
+	epf_test->bar_size[_id] = val;					\
+									\
+	return len;							\
+}
+
+PCI_EPF_TEST_BAR_SIZE_R(bar0_size, BAR_0)
+PCI_EPF_TEST_BAR_SIZE_W(bar0_size, BAR_0)
+PCI_EPF_TEST_BAR_SIZE_R(bar1_size, BAR_1)
+PCI_EPF_TEST_BAR_SIZE_W(bar1_size, BAR_1)
+PCI_EPF_TEST_BAR_SIZE_R(bar2_size, BAR_2)
+PCI_EPF_TEST_BAR_SIZE_W(bar2_size, BAR_2)
+PCI_EPF_TEST_BAR_SIZE_R(bar3_size, BAR_3)
+PCI_EPF_TEST_BAR_SIZE_W(bar3_size, BAR_3)
+PCI_EPF_TEST_BAR_SIZE_R(bar4_size, BAR_4)
+PCI_EPF_TEST_BAR_SIZE_W(bar4_size, BAR_4)
+PCI_EPF_TEST_BAR_SIZE_R(bar5_size, BAR_5)
+PCI_EPF_TEST_BAR_SIZE_W(bar5_size, BAR_5)
+
+CONFIGFS_ATTR(pci_epf_test_, bar0_size);
+CONFIGFS_ATTR(pci_epf_test_, bar1_size);
+CONFIGFS_ATTR(pci_epf_test_, bar2_size);
+CONFIGFS_ATTR(pci_epf_test_, bar3_size);
+CONFIGFS_ATTR(pci_epf_test_, bar4_size);
+CONFIGFS_ATTR(pci_epf_test_, bar5_size);
+
+static struct configfs_attribute *pci_epf_test_attrs[] = {
+	&pci_epf_test_attr_bar0_size,
+	&pci_epf_test_attr_bar1_size,
+	&pci_epf_test_attr_bar2_size,
+	&pci_epf_test_attr_bar3_size,
+	&pci_epf_test_attr_bar4_size,
+	&pci_epf_test_attr_bar5_size,
+	NULL,
+};
+
+static const struct config_item_type pci_epf_test_group_type = {
+	.ct_attrs	= pci_epf_test_attrs,
+	.ct_owner	= THIS_MODULE,
+};
+
+static struct config_group *pci_epf_test_add_cfs(struct pci_epf *epf,
+						 struct config_group *group)
+{
+	struct pci_epf_test *epf_test = epf_get_drvdata(epf);
+	struct config_group *epf_group = &epf_test->group;
+	struct device *dev = &epf->dev;
+
+	config_group_init_type_name(epf_group, dev_name(dev),
+				    &pci_epf_test_group_type);
+
+	return epf_group;
+}
+
 static const struct pci_epf_device_id pci_epf_test_ids[] = {
 	{
 		.name = "pci_epf_test",
@@ -1154,6 +1415,7 @@ static int pci_epf_test_probe(struct pci_epf *epf,
 {
 	struct pci_epf_test *epf_test;
 	struct device *dev = &epf->dev;
+	enum pci_barno bar;
 
 	epf_test = devm_kzalloc(dev, sizeof(*epf_test), GFP_KERNEL);
 	if (!epf_test)
@@ -1161,6 +1423,8 @@ static int pci_epf_test_probe(struct pci_epf *epf,
 
 	epf->header = &test_header;
 	epf_test->epf = epf;
+	for (bar = BAR_0; bar < PCI_STD_NUM_BARS; bar++)
+		epf_test->bar_size[bar] = default_bar_size[bar];
 
 	INIT_DELAYED_WORK(&epf_test->cmd_handler, pci_epf_test_cmd_handler);
 
@@ -1173,6 +1437,7 @@ static int pci_epf_test_probe(struct pci_epf *epf,
 static const struct pci_epf_ops ops = {
 	.unbind	= pci_epf_test_unbind,
 	.bind	= pci_epf_test_bind,
+	.add_cfs = pci_epf_test_add_cfs,
 };
 
 static struct pci_epf_driver test_driver = {