11da177e4SLinus Torvalds /* 21da177e4SLinus Torvalds * I/O SAPIC support. 31da177e4SLinus Torvalds * 41da177e4SLinus Torvalds * Copyright (C) 1999 Intel Corp. 51da177e4SLinus Torvalds * Copyright (C) 1999 Asit Mallick <asit.k.mallick@intel.com> 61da177e4SLinus Torvalds * Copyright (C) 2000-2002 J.I. Lee <jung-ik.lee@intel.com> 71da177e4SLinus Torvalds * Copyright (C) 1999-2000, 2002-2003 Hewlett-Packard Co. 81da177e4SLinus Torvalds * David Mosberger-Tang <davidm@hpl.hp.com> 91da177e4SLinus Torvalds * Copyright (C) 1999 VA Linux Systems 101da177e4SLinus Torvalds * Copyright (C) 1999,2000 Walt Drummond <drummond@valinux.com> 111da177e4SLinus Torvalds * 1246cba3dcSSatoru Takeuchi * 00/04/19 D. Mosberger Rewritten to mirror more closely the x86 I/O 1346cba3dcSSatoru Takeuchi * APIC code. In particular, we now have separate 1446cba3dcSSatoru Takeuchi * handlers for edge and level triggered 1546cba3dcSSatoru Takeuchi * interrupts. 1646cba3dcSSatoru Takeuchi * 00/10/27 Asit Mallick, Goutham Rao <goutham.rao@intel.com> IRQ vector 1746cba3dcSSatoru Takeuchi * allocation PCI to vector mapping, shared PCI 1846cba3dcSSatoru Takeuchi * interrupts. 1946cba3dcSSatoru Takeuchi * 00/10/27 D. Mosberger Document things a bit more to make them more 2046cba3dcSSatoru Takeuchi * understandable. Clean up much of the old 2146cba3dcSSatoru Takeuchi * IOSAPIC cruft. 2246cba3dcSSatoru Takeuchi * 01/07/27 J.I. Lee PCI irq routing, Platform/Legacy interrupts 2346cba3dcSSatoru Takeuchi * and fixes for ACPI S5(SoftOff) support. 241da177e4SLinus Torvalds * 02/01/23 J.I. Lee iosapic pgm fixes for PCI irq routing from _PRT 2546cba3dcSSatoru Takeuchi * 02/01/07 E. Focht <efocht@ess.nec.de> Redirectable interrupt 2646cba3dcSSatoru Takeuchi * vectors in iosapic_set_affinity(), 2746cba3dcSSatoru Takeuchi * initializations for /proc/irq/#/smp_affinity 281da177e4SLinus Torvalds * 02/04/02 P. Diefenbaugh Cleaned up ACPI PCI IRQ routing. 291da177e4SLinus Torvalds * 02/04/18 J.I. Lee bug fix in iosapic_init_pci_irq 3046cba3dcSSatoru Takeuchi * 02/04/30 J.I. Lee bug fix in find_iosapic to fix ACPI PCI IRQ to 3146cba3dcSSatoru Takeuchi * IOSAPIC mapping error 321da177e4SLinus Torvalds * 02/07/29 T. Kochi Allocate interrupt vectors dynamically 3346cba3dcSSatoru Takeuchi * 02/08/04 T. Kochi Cleaned up terminology (irq, global system 3446cba3dcSSatoru Takeuchi * interrupt, vector, etc.) 3546cba3dcSSatoru Takeuchi * 02/09/20 D. Mosberger Simplified by taking advantage of ACPI's 3646cba3dcSSatoru Takeuchi * pci_irq code. 371da177e4SLinus Torvalds * 03/02/19 B. Helgaas Make pcat_compat system-wide, not per-IOSAPIC. 3846cba3dcSSatoru Takeuchi * Remove iosapic_address & gsi_base from 3946cba3dcSSatoru Takeuchi * external interfaces. Rationalize 4046cba3dcSSatoru Takeuchi * __init/__devinit attributes. 411da177e4SLinus Torvalds * 04/12/04 Ashok Raj <ashok.raj@intel.com> Intel Corporation 2004 4246cba3dcSSatoru Takeuchi * Updated to work with irq migration necessary 4346cba3dcSSatoru Takeuchi * for CPU Hotplug 441da177e4SLinus Torvalds */ 451da177e4SLinus Torvalds /* 4646cba3dcSSatoru Takeuchi * Here is what the interrupt logic between a PCI device and the kernel looks 4746cba3dcSSatoru Takeuchi * like: 481da177e4SLinus Torvalds * 4946cba3dcSSatoru Takeuchi * (1) A PCI device raises one of the four interrupt pins (INTA, INTB, INTC, 5046cba3dcSSatoru Takeuchi * INTD). The device is uniquely identified by its bus-, and slot-number 5146cba3dcSSatoru Takeuchi * (the function number does not matter here because all functions share 5246cba3dcSSatoru Takeuchi * the same interrupt lines). 531da177e4SLinus Torvalds * 5446cba3dcSSatoru Takeuchi * (2) The motherboard routes the interrupt line to a pin on a IOSAPIC 5546cba3dcSSatoru Takeuchi * controller. Multiple interrupt lines may have to share the same 5646cba3dcSSatoru Takeuchi * IOSAPIC pin (if they're level triggered and use the same polarity). 5746cba3dcSSatoru Takeuchi * Each interrupt line has a unique Global System Interrupt (GSI) number 5846cba3dcSSatoru Takeuchi * which can be calculated as the sum of the controller's base GSI number 5946cba3dcSSatoru Takeuchi * and the IOSAPIC pin number to which the line connects. 601da177e4SLinus Torvalds * 6146cba3dcSSatoru Takeuchi * (3) The IOSAPIC uses an internal routing table entries (RTEs) to map the 6246cba3dcSSatoru Takeuchi * IOSAPIC pin into the IA-64 interrupt vector. This interrupt vector is then 6346cba3dcSSatoru Takeuchi * sent to the CPU. 641da177e4SLinus Torvalds * 6546cba3dcSSatoru Takeuchi * (4) The kernel recognizes an interrupt as an IRQ. The IRQ interface is 6646cba3dcSSatoru Takeuchi * used as architecture-independent interrupt handling mechanism in Linux. 6746cba3dcSSatoru Takeuchi * As an IRQ is a number, we have to have 6846cba3dcSSatoru Takeuchi * IA-64 interrupt vector number <-> IRQ number mapping. On smaller 6946cba3dcSSatoru Takeuchi * systems, we use one-to-one mapping between IA-64 vector and IRQ. A 7046cba3dcSSatoru Takeuchi * platform can implement platform_irq_to_vector(irq) and 711da177e4SLinus Torvalds * platform_local_vector_to_irq(vector) APIs to differentiate the mapping. 727f30491cSTony Luck * Please see also arch/ia64/include/asm/hw_irq.h for those APIs. 731da177e4SLinus Torvalds * 741da177e4SLinus Torvalds * To sum up, there are three levels of mappings involved: 751da177e4SLinus Torvalds * 761da177e4SLinus Torvalds * PCI pin -> global system interrupt (GSI) -> IA-64 vector <-> IRQ 771da177e4SLinus Torvalds * 7846cba3dcSSatoru Takeuchi * Note: The term "IRQ" is loosely used everywhere in Linux kernel to 7946cba3dcSSatoru Takeuchi * describe interrupts. Now we use "IRQ" only for Linux IRQ's. ISA IRQ 8046cba3dcSSatoru Takeuchi * (isa_irq) is the only exception in this source code. 811da177e4SLinus Torvalds */ 821da177e4SLinus Torvalds 831da177e4SLinus Torvalds #include <linux/acpi.h> 841da177e4SLinus Torvalds #include <linux/init.h> 851da177e4SLinus Torvalds #include <linux/irq.h> 861da177e4SLinus Torvalds #include <linux/kernel.h> 871da177e4SLinus Torvalds #include <linux/list.h> 881da177e4SLinus Torvalds #include <linux/pci.h> 895a0e3ad6STejun Heo #include <linux/slab.h> 901da177e4SLinus Torvalds #include <linux/smp.h> 911da177e4SLinus Torvalds #include <linux/string.h> 9224eeb568SKenji Kaneshige #include <linux/bootmem.h> 931da177e4SLinus Torvalds 941da177e4SLinus Torvalds #include <asm/delay.h> 951da177e4SLinus Torvalds #include <asm/hw_irq.h> 961da177e4SLinus Torvalds #include <asm/io.h> 971da177e4SLinus Torvalds #include <asm/iosapic.h> 981da177e4SLinus Torvalds #include <asm/machvec.h> 991da177e4SLinus Torvalds #include <asm/processor.h> 1001da177e4SLinus Torvalds #include <asm/ptrace.h> 1011da177e4SLinus Torvalds 1021da177e4SLinus Torvalds #undef DEBUG_INTERRUPT_ROUTING 1031da177e4SLinus Torvalds 1041da177e4SLinus Torvalds #ifdef DEBUG_INTERRUPT_ROUTING 1051da177e4SLinus Torvalds #define DBG(fmt...) printk(fmt) 1061da177e4SLinus Torvalds #else 1071da177e4SLinus Torvalds #define DBG(fmt...) 1081da177e4SLinus Torvalds #endif 1091da177e4SLinus Torvalds 1101da177e4SLinus Torvalds static DEFINE_SPINLOCK(iosapic_lock); 1111da177e4SLinus Torvalds 11246cba3dcSSatoru Takeuchi /* 11346cba3dcSSatoru Takeuchi * These tables map IA-64 vectors to the IOSAPIC pin that generates this 11446cba3dcSSatoru Takeuchi * vector. 11546cba3dcSSatoru Takeuchi */ 116e1b30a39SYasuaki Ishimatsu 117e1b30a39SYasuaki Ishimatsu #define NO_REF_RTE 0 118e1b30a39SYasuaki Ishimatsu 119c5e3f9e5SYasuaki Ishimatsu static struct iosapic { 120c5e3f9e5SYasuaki Ishimatsu char __iomem *addr; /* base address of IOSAPIC */ 121c5e3f9e5SYasuaki Ishimatsu unsigned int gsi_base; /* GSI base */ 122c5e3f9e5SYasuaki Ishimatsu unsigned short num_rte; /* # of RTEs on this IOSAPIC */ 123c5e3f9e5SYasuaki Ishimatsu int rtes_inuse; /* # of RTEs in use on this IOSAPIC */ 124c5e3f9e5SYasuaki Ishimatsu #ifdef CONFIG_NUMA 125c5e3f9e5SYasuaki Ishimatsu unsigned short node; /* numa node association via pxm */ 126c5e3f9e5SYasuaki Ishimatsu #endif 127c1726d6fSYasuaki Ishimatsu spinlock_t lock; /* lock for indirect reg access */ 128c5e3f9e5SYasuaki Ishimatsu } iosapic_lists[NR_IOSAPICS]; 1291da177e4SLinus Torvalds 13024eeb568SKenji Kaneshige struct iosapic_rte_info { 131c5e3f9e5SYasuaki Ishimatsu struct list_head rte_list; /* RTEs sharing the same vector */ 13224eeb568SKenji Kaneshige char rte_index; /* IOSAPIC RTE index */ 13324eeb568SKenji Kaneshige int refcnt; /* reference counter */ 134c5e3f9e5SYasuaki Ishimatsu struct iosapic *iosapic; 13524eeb568SKenji Kaneshige } ____cacheline_aligned; 13624eeb568SKenji Kaneshige 13724eeb568SKenji Kaneshige static struct iosapic_intr_info { 13846cba3dcSSatoru Takeuchi struct list_head rtes; /* RTEs using this vector (empty => 13946cba3dcSSatoru Takeuchi * not an IOSAPIC interrupt) */ 140c4c376f7SKenji Kaneshige int count; /* # of registered RTEs */ 14146cba3dcSSatoru Takeuchi u32 low32; /* current value of low word of 14246cba3dcSSatoru Takeuchi * Redirection table entry */ 14324eeb568SKenji Kaneshige unsigned int dest; /* destination CPU physical ID */ 1441da177e4SLinus Torvalds unsigned char dmode : 3; /* delivery mode (see iosapic.h) */ 14546cba3dcSSatoru Takeuchi unsigned char polarity: 1; /* interrupt polarity 14646cba3dcSSatoru Takeuchi * (see iosapic.h) */ 1471da177e4SLinus Torvalds unsigned char trigger : 1; /* trigger mode (see iosapic.h) */ 1484bbdec7aSYasuaki Ishimatsu } iosapic_intr_info[NR_IRQS]; 1491da177e4SLinus Torvalds 1505b5e76e9SGreg Kroah-Hartman static unsigned char pcat_compat; /* 8259 compatibility flag */ 1511da177e4SLinus Torvalds 152c1726d6fSYasuaki Ishimatsu static inline void 153c1726d6fSYasuaki Ishimatsu iosapic_write(struct iosapic *iosapic, unsigned int reg, u32 val) 154c1726d6fSYasuaki Ishimatsu { 155c1726d6fSYasuaki Ishimatsu unsigned long flags; 156c1726d6fSYasuaki Ishimatsu 157c1726d6fSYasuaki Ishimatsu spin_lock_irqsave(&iosapic->lock, flags); 158c1726d6fSYasuaki Ishimatsu __iosapic_write(iosapic->addr, reg, val); 159c1726d6fSYasuaki Ishimatsu spin_unlock_irqrestore(&iosapic->lock, flags); 160c1726d6fSYasuaki Ishimatsu } 161c1726d6fSYasuaki Ishimatsu 1621da177e4SLinus Torvalds /* 1631da177e4SLinus Torvalds * Find an IOSAPIC associated with a GSI 1641da177e4SLinus Torvalds */ 1651da177e4SLinus Torvalds static inline int 1661da177e4SLinus Torvalds find_iosapic (unsigned int gsi) 1671da177e4SLinus Torvalds { 1681da177e4SLinus Torvalds int i; 1691da177e4SLinus Torvalds 1700e888adcSKenji Kaneshige for (i = 0; i < NR_IOSAPICS; i++) { 17146cba3dcSSatoru Takeuchi if ((unsigned) (gsi - iosapic_lists[i].gsi_base) < 17246cba3dcSSatoru Takeuchi iosapic_lists[i].num_rte) 1731da177e4SLinus Torvalds return i; 1741da177e4SLinus Torvalds } 1751da177e4SLinus Torvalds 1761da177e4SLinus Torvalds return -1; 1771da177e4SLinus Torvalds } 1781da177e4SLinus Torvalds 1794bbdec7aSYasuaki Ishimatsu static inline int __gsi_to_irq(unsigned int gsi) 1801da177e4SLinus Torvalds { 1814bbdec7aSYasuaki Ishimatsu int irq; 1821da177e4SLinus Torvalds struct iosapic_intr_info *info; 18324eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 1841da177e4SLinus Torvalds 1854bbdec7aSYasuaki Ishimatsu for (irq = 0; irq < NR_IRQS; irq++) { 1864bbdec7aSYasuaki Ishimatsu info = &iosapic_intr_info[irq]; 18724eeb568SKenji Kaneshige list_for_each_entry(rte, &info->rtes, rte_list) 188c5e3f9e5SYasuaki Ishimatsu if (rte->iosapic->gsi_base + rte->rte_index == gsi) 1894bbdec7aSYasuaki Ishimatsu return irq; 1904bbdec7aSYasuaki Ishimatsu } 1911da177e4SLinus Torvalds return -1; 1921da177e4SLinus Torvalds } 1931da177e4SLinus Torvalds 1941da177e4SLinus Torvalds int 1951da177e4SLinus Torvalds gsi_to_irq (unsigned int gsi) 1961da177e4SLinus Torvalds { 19724eeb568SKenji Kaneshige unsigned long flags; 19824eeb568SKenji Kaneshige int irq; 19924eeb568SKenji Kaneshige 2004bbdec7aSYasuaki Ishimatsu spin_lock_irqsave(&iosapic_lock, flags); 2014bbdec7aSYasuaki Ishimatsu irq = __gsi_to_irq(gsi); 2024bbdec7aSYasuaki Ishimatsu spin_unlock_irqrestore(&iosapic_lock, flags); 20324eeb568SKenji Kaneshige return irq; 20424eeb568SKenji Kaneshige } 20524eeb568SKenji Kaneshige 2064bbdec7aSYasuaki Ishimatsu static struct iosapic_rte_info *find_rte(unsigned int irq, unsigned int gsi) 20724eeb568SKenji Kaneshige { 20824eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 20924eeb568SKenji Kaneshige 2104bbdec7aSYasuaki Ishimatsu list_for_each_entry(rte, &iosapic_intr_info[irq].rtes, rte_list) 211c5e3f9e5SYasuaki Ishimatsu if (rte->iosapic->gsi_base + rte->rte_index == gsi) 21224eeb568SKenji Kaneshige return rte; 21324eeb568SKenji Kaneshige return NULL; 2141da177e4SLinus Torvalds } 2151da177e4SLinus Torvalds 2161da177e4SLinus Torvalds static void 2174bbdec7aSYasuaki Ishimatsu set_rte (unsigned int gsi, unsigned int irq, unsigned int dest, int mask) 2181da177e4SLinus Torvalds { 2191da177e4SLinus Torvalds unsigned long pol, trigger, dmode; 2201da177e4SLinus Torvalds u32 low32, high32; 2211da177e4SLinus Torvalds int rte_index; 2221da177e4SLinus Torvalds char redir; 22324eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 2244bbdec7aSYasuaki Ishimatsu ia64_vector vector = irq_to_vector(irq); 2251da177e4SLinus Torvalds 2261da177e4SLinus Torvalds DBG(KERN_DEBUG"IOSAPIC: routing vector %d to 0x%x\n", vector, dest); 2271da177e4SLinus Torvalds 2284bbdec7aSYasuaki Ishimatsu rte = find_rte(irq, gsi); 22924eeb568SKenji Kaneshige if (!rte) 2301da177e4SLinus Torvalds return; /* not an IOSAPIC interrupt */ 2311da177e4SLinus Torvalds 23224eeb568SKenji Kaneshige rte_index = rte->rte_index; 2334bbdec7aSYasuaki Ishimatsu pol = iosapic_intr_info[irq].polarity; 2344bbdec7aSYasuaki Ishimatsu trigger = iosapic_intr_info[irq].trigger; 2354bbdec7aSYasuaki Ishimatsu dmode = iosapic_intr_info[irq].dmode; 2361da177e4SLinus Torvalds 2371da177e4SLinus Torvalds redir = (dmode == IOSAPIC_LOWEST_PRIORITY) ? 1 : 0; 2381da177e4SLinus Torvalds 2391da177e4SLinus Torvalds #ifdef CONFIG_SMP 2404bbdec7aSYasuaki Ishimatsu set_irq_affinity_info(irq, (int)(dest & 0xffff), redir); 2411da177e4SLinus Torvalds #endif 2421da177e4SLinus Torvalds 2431da177e4SLinus Torvalds low32 = ((pol << IOSAPIC_POLARITY_SHIFT) | 2441da177e4SLinus Torvalds (trigger << IOSAPIC_TRIGGER_SHIFT) | 2451da177e4SLinus Torvalds (dmode << IOSAPIC_DELIVERY_SHIFT) | 2461da177e4SLinus Torvalds ((mask ? 1 : 0) << IOSAPIC_MASK_SHIFT) | 2471da177e4SLinus Torvalds vector); 2481da177e4SLinus Torvalds 2491da177e4SLinus Torvalds /* dest contains both id and eid */ 2501da177e4SLinus Torvalds high32 = (dest << IOSAPIC_DEST_SHIFT); 2511da177e4SLinus Torvalds 252c1726d6fSYasuaki Ishimatsu iosapic_write(rte->iosapic, IOSAPIC_RTE_HIGH(rte_index), high32); 253c1726d6fSYasuaki Ishimatsu iosapic_write(rte->iosapic, IOSAPIC_RTE_LOW(rte_index), low32); 2544bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].low32 = low32; 2554bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].dest = dest; 2561da177e4SLinus Torvalds } 2571da177e4SLinus Torvalds 2581da177e4SLinus Torvalds static void 2598fac171fSThomas Gleixner nop (struct irq_data *data) 2601da177e4SLinus Torvalds { 2611da177e4SLinus Torvalds /* do nothing... */ 2621da177e4SLinus Torvalds } 2631da177e4SLinus Torvalds 264a7956113SZou Nan hai 265a7956113SZou Nan hai #ifdef CONFIG_KEXEC 266a7956113SZou Nan hai void 267a7956113SZou Nan hai kexec_disable_iosapic(void) 268a7956113SZou Nan hai { 269a7956113SZou Nan hai struct iosapic_intr_info *info; 270a7956113SZou Nan hai struct iosapic_rte_info *rte; 2714bbdec7aSYasuaki Ishimatsu ia64_vector vec; 2724bbdec7aSYasuaki Ishimatsu int irq; 2734bbdec7aSYasuaki Ishimatsu 2744bbdec7aSYasuaki Ishimatsu for (irq = 0; irq < NR_IRQS; irq++) { 2754bbdec7aSYasuaki Ishimatsu info = &iosapic_intr_info[irq]; 2764bbdec7aSYasuaki Ishimatsu vec = irq_to_vector(irq); 277a7956113SZou Nan hai list_for_each_entry(rte, &info->rtes, 278a7956113SZou Nan hai rte_list) { 279c1726d6fSYasuaki Ishimatsu iosapic_write(rte->iosapic, 280a7956113SZou Nan hai IOSAPIC_RTE_LOW(rte->rte_index), 281a7956113SZou Nan hai IOSAPIC_MASK|vec); 282c5e3f9e5SYasuaki Ishimatsu iosapic_eoi(rte->iosapic->addr, vec); 283a7956113SZou Nan hai } 284a7956113SZou Nan hai } 285a7956113SZou Nan hai } 286a7956113SZou Nan hai #endif 287a7956113SZou Nan hai 2881da177e4SLinus Torvalds static void 2898fac171fSThomas Gleixner mask_irq (struct irq_data *data) 2901da177e4SLinus Torvalds { 2918fac171fSThomas Gleixner unsigned int irq = data->irq; 2921da177e4SLinus Torvalds u32 low32; 2931da177e4SLinus Torvalds int rte_index; 29424eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 2951da177e4SLinus Torvalds 296c4c376f7SKenji Kaneshige if (!iosapic_intr_info[irq].count) 2971da177e4SLinus Torvalds return; /* not an IOSAPIC interrupt! */ 2981da177e4SLinus Torvalds 2991da177e4SLinus Torvalds /* set only the mask bit */ 3004bbdec7aSYasuaki Ishimatsu low32 = iosapic_intr_info[irq].low32 |= IOSAPIC_MASK; 3014bbdec7aSYasuaki Ishimatsu list_for_each_entry(rte, &iosapic_intr_info[irq].rtes, rte_list) { 30224eeb568SKenji Kaneshige rte_index = rte->rte_index; 303c1726d6fSYasuaki Ishimatsu iosapic_write(rte->iosapic, IOSAPIC_RTE_LOW(rte_index), low32); 3041da177e4SLinus Torvalds } 3051da177e4SLinus Torvalds } 3061da177e4SLinus Torvalds 3071da177e4SLinus Torvalds static void 3088fac171fSThomas Gleixner unmask_irq (struct irq_data *data) 3091da177e4SLinus Torvalds { 3108fac171fSThomas Gleixner unsigned int irq = data->irq; 3111da177e4SLinus Torvalds u32 low32; 3121da177e4SLinus Torvalds int rte_index; 31324eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 3141da177e4SLinus Torvalds 315c4c376f7SKenji Kaneshige if (!iosapic_intr_info[irq].count) 3161da177e4SLinus Torvalds return; /* not an IOSAPIC interrupt! */ 3171da177e4SLinus Torvalds 3184bbdec7aSYasuaki Ishimatsu low32 = iosapic_intr_info[irq].low32 &= ~IOSAPIC_MASK; 3194bbdec7aSYasuaki Ishimatsu list_for_each_entry(rte, &iosapic_intr_info[irq].rtes, rte_list) { 32024eeb568SKenji Kaneshige rte_index = rte->rte_index; 321c1726d6fSYasuaki Ishimatsu iosapic_write(rte->iosapic, IOSAPIC_RTE_LOW(rte_index), low32); 3221da177e4SLinus Torvalds } 3231da177e4SLinus Torvalds } 3241da177e4SLinus Torvalds 3251da177e4SLinus Torvalds 326d5dedd45SYinghai Lu static int 3278fac171fSThomas Gleixner iosapic_set_affinity(struct irq_data *data, const struct cpumask *mask, 3288fac171fSThomas Gleixner bool force) 3291da177e4SLinus Torvalds { 3301da177e4SLinus Torvalds #ifdef CONFIG_SMP 3318fac171fSThomas Gleixner unsigned int irq = data->irq; 3321da177e4SLinus Torvalds u32 high32, low32; 3330de26520SRusty Russell int cpu, dest, rte_index; 3341da177e4SLinus Torvalds int redir = (irq & IA64_IRQ_REDIRECTED) ? 1 : 0; 33524eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 336c1726d6fSYasuaki Ishimatsu struct iosapic *iosapic; 3371da177e4SLinus Torvalds 3381da177e4SLinus Torvalds irq &= (~IA64_IRQ_REDIRECTED); 3391da177e4SLinus Torvalds 3400de26520SRusty Russell cpu = cpumask_first_and(cpu_online_mask, mask); 3410de26520SRusty Russell if (cpu >= nr_cpu_ids) 342d5dedd45SYinghai Lu return -1; 3431da177e4SLinus Torvalds 3440de26520SRusty Russell if (irq_prepare_move(irq, cpu)) 345d5dedd45SYinghai Lu return -1; 346cd378f18SYasuaki Ishimatsu 3470de26520SRusty Russell dest = cpu_physical_id(cpu); 3481da177e4SLinus Torvalds 349c4c376f7SKenji Kaneshige if (!iosapic_intr_info[irq].count) 350d5dedd45SYinghai Lu return -1; /* not an IOSAPIC interrupt */ 3511da177e4SLinus Torvalds 3521da177e4SLinus Torvalds set_irq_affinity_info(irq, dest, redir); 3531da177e4SLinus Torvalds 3541da177e4SLinus Torvalds /* dest contains both id and eid */ 3551da177e4SLinus Torvalds high32 = dest << IOSAPIC_DEST_SHIFT; 3561da177e4SLinus Torvalds 3574bbdec7aSYasuaki Ishimatsu low32 = iosapic_intr_info[irq].low32 & ~(7 << IOSAPIC_DELIVERY_SHIFT); 3581da177e4SLinus Torvalds if (redir) 3591da177e4SLinus Torvalds /* change delivery mode to lowest priority */ 360e3a8f7b8SYasuaki Ishimatsu low32 |= (IOSAPIC_LOWEST_PRIORITY << IOSAPIC_DELIVERY_SHIFT); 3611da177e4SLinus Torvalds else 3621da177e4SLinus Torvalds /* change delivery mode to fixed */ 3631da177e4SLinus Torvalds low32 |= (IOSAPIC_FIXED << IOSAPIC_DELIVERY_SHIFT); 364cd378f18SYasuaki Ishimatsu low32 &= IOSAPIC_VECTOR_MASK; 365cd378f18SYasuaki Ishimatsu low32 |= irq_to_vector(irq); 3661da177e4SLinus Torvalds 3674bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].low32 = low32; 3684bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].dest = dest; 3694bbdec7aSYasuaki Ishimatsu list_for_each_entry(rte, &iosapic_intr_info[irq].rtes, rte_list) { 370c1726d6fSYasuaki Ishimatsu iosapic = rte->iosapic; 37124eeb568SKenji Kaneshige rte_index = rte->rte_index; 372c1726d6fSYasuaki Ishimatsu iosapic_write(iosapic, IOSAPIC_RTE_HIGH(rte_index), high32); 373c1726d6fSYasuaki Ishimatsu iosapic_write(iosapic, IOSAPIC_RTE_LOW(rte_index), low32); 3741da177e4SLinus Torvalds } 375d5dedd45SYinghai Lu 3761da177e4SLinus Torvalds #endif 377d5dedd45SYinghai Lu return 0; 3781da177e4SLinus Torvalds } 3791da177e4SLinus Torvalds 3801da177e4SLinus Torvalds /* 3811da177e4SLinus Torvalds * Handlers for level-triggered interrupts. 3821da177e4SLinus Torvalds */ 3831da177e4SLinus Torvalds 3841da177e4SLinus Torvalds static unsigned int 3858fac171fSThomas Gleixner iosapic_startup_level_irq (struct irq_data *data) 3861da177e4SLinus Torvalds { 3878fac171fSThomas Gleixner unmask_irq(data); 3881da177e4SLinus Torvalds return 0; 3891da177e4SLinus Torvalds } 3901da177e4SLinus Torvalds 3911da177e4SLinus Torvalds static void 3928fac171fSThomas Gleixner iosapic_unmask_level_irq (struct irq_data *data) 3931da177e4SLinus Torvalds { 3948fac171fSThomas Gleixner unsigned int irq = data->irq; 3951da177e4SLinus Torvalds ia64_vector vec = irq_to_vector(irq); 39624eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 397cd378f18SYasuaki Ishimatsu int do_unmask_irq = 0; 3981da177e4SLinus Torvalds 399a6cd6322SKenji Kaneshige irq_complete_move(irq); 40091ce72e0SThomas Gleixner if (unlikely(irqd_is_setaffinity_pending(data))) { 401cd378f18SYasuaki Ishimatsu do_unmask_irq = 1; 4028fac171fSThomas Gleixner mask_irq(data); 4035d4bff94STony Luck } else 4048fac171fSThomas Gleixner unmask_irq(data); 405cd378f18SYasuaki Ishimatsu 4064bbdec7aSYasuaki Ishimatsu list_for_each_entry(rte, &iosapic_intr_info[irq].rtes, rte_list) 407c5e3f9e5SYasuaki Ishimatsu iosapic_eoi(rte->iosapic->addr, vec); 408cd378f18SYasuaki Ishimatsu 409cd378f18SYasuaki Ishimatsu if (unlikely(do_unmask_irq)) { 41091ce72e0SThomas Gleixner irq_move_masked_irq(data); 4118fac171fSThomas Gleixner unmask_irq(data); 412cd378f18SYasuaki Ishimatsu } 4131da177e4SLinus Torvalds } 4141da177e4SLinus Torvalds 4151da177e4SLinus Torvalds #define iosapic_shutdown_level_irq mask_irq 4161da177e4SLinus Torvalds #define iosapic_enable_level_irq unmask_irq 4171da177e4SLinus Torvalds #define iosapic_disable_level_irq mask_irq 4181da177e4SLinus Torvalds #define iosapic_ack_level_irq nop 4191da177e4SLinus Torvalds 4209e004ebdSSimon Horman static struct irq_chip irq_type_iosapic_level = { 42106344db3SIngo Molnar .name = "IO-SAPIC-level", 4228fac171fSThomas Gleixner .irq_startup = iosapic_startup_level_irq, 4238fac171fSThomas Gleixner .irq_shutdown = iosapic_shutdown_level_irq, 4248fac171fSThomas Gleixner .irq_enable = iosapic_enable_level_irq, 4258fac171fSThomas Gleixner .irq_disable = iosapic_disable_level_irq, 4268fac171fSThomas Gleixner .irq_ack = iosapic_ack_level_irq, 4278fac171fSThomas Gleixner .irq_mask = mask_irq, 4288fac171fSThomas Gleixner .irq_unmask = iosapic_unmask_level_irq, 4298fac171fSThomas Gleixner .irq_set_affinity = iosapic_set_affinity 4301da177e4SLinus Torvalds }; 4311da177e4SLinus Torvalds 4321da177e4SLinus Torvalds /* 4331da177e4SLinus Torvalds * Handlers for edge-triggered interrupts. 4341da177e4SLinus Torvalds */ 4351da177e4SLinus Torvalds 4361da177e4SLinus Torvalds static unsigned int 4378fac171fSThomas Gleixner iosapic_startup_edge_irq (struct irq_data *data) 4381da177e4SLinus Torvalds { 4398fac171fSThomas Gleixner unmask_irq(data); 4401da177e4SLinus Torvalds /* 4411da177e4SLinus Torvalds * IOSAPIC simply drops interrupts pended while the 4421da177e4SLinus Torvalds * corresponding pin was masked, so we can't know if an 4431da177e4SLinus Torvalds * interrupt is pending already. Let's hope not... 4441da177e4SLinus Torvalds */ 4451da177e4SLinus Torvalds return 0; 4461da177e4SLinus Torvalds } 4471da177e4SLinus Torvalds 4481da177e4SLinus Torvalds static void 4498fac171fSThomas Gleixner iosapic_ack_edge_irq (struct irq_data *data) 4501da177e4SLinus Torvalds { 45191ce72e0SThomas Gleixner irq_complete_move(data->irq); 45291ce72e0SThomas Gleixner irq_move_irq(data); 4531da177e4SLinus Torvalds } 4541da177e4SLinus Torvalds 4551da177e4SLinus Torvalds #define iosapic_enable_edge_irq unmask_irq 4561da177e4SLinus Torvalds #define iosapic_disable_edge_irq nop 4571da177e4SLinus Torvalds 4589e004ebdSSimon Horman static struct irq_chip irq_type_iosapic_edge = { 45906344db3SIngo Molnar .name = "IO-SAPIC-edge", 4608fac171fSThomas Gleixner .irq_startup = iosapic_startup_edge_irq, 4618fac171fSThomas Gleixner .irq_shutdown = iosapic_disable_edge_irq, 4628fac171fSThomas Gleixner .irq_enable = iosapic_enable_edge_irq, 4638fac171fSThomas Gleixner .irq_disable = iosapic_disable_edge_irq, 4648fac171fSThomas Gleixner .irq_ack = iosapic_ack_edge_irq, 4658fac171fSThomas Gleixner .irq_mask = mask_irq, 4668fac171fSThomas Gleixner .irq_unmask = unmask_irq, 4678fac171fSThomas Gleixner .irq_set_affinity = iosapic_set_affinity 4681da177e4SLinus Torvalds }; 4691da177e4SLinus Torvalds 4709e004ebdSSimon Horman static unsigned int 4711da177e4SLinus Torvalds iosapic_version (char __iomem *addr) 4721da177e4SLinus Torvalds { 4731da177e4SLinus Torvalds /* 4741da177e4SLinus Torvalds * IOSAPIC Version Register return 32 bit structure like: 4751da177e4SLinus Torvalds * { 4761da177e4SLinus Torvalds * unsigned int version : 8; 4771da177e4SLinus Torvalds * unsigned int reserved1 : 8; 4781da177e4SLinus Torvalds * unsigned int max_redir : 8; 4791da177e4SLinus Torvalds * unsigned int reserved2 : 8; 4801da177e4SLinus Torvalds * } 4811da177e4SLinus Torvalds */ 482c1726d6fSYasuaki Ishimatsu return __iosapic_read(addr, IOSAPIC_VERSION); 4831da177e4SLinus Torvalds } 4841da177e4SLinus Torvalds 4854bbdec7aSYasuaki Ishimatsu static int iosapic_find_sharable_irq(unsigned long trigger, unsigned long pol) 48624eeb568SKenji Kaneshige { 4874bbdec7aSYasuaki Ishimatsu int i, irq = -ENOSPC, min_count = -1; 48824eeb568SKenji Kaneshige struct iosapic_intr_info *info; 48924eeb568SKenji Kaneshige 49024eeb568SKenji Kaneshige /* 49124eeb568SKenji Kaneshige * shared vectors for edge-triggered interrupts are not 49224eeb568SKenji Kaneshige * supported yet 49324eeb568SKenji Kaneshige */ 49424eeb568SKenji Kaneshige if (trigger == IOSAPIC_EDGE) 49540598cbeSYasuaki Ishimatsu return -EINVAL; 49624eeb568SKenji Kaneshige 4975b592397SRoel Kluin for (i = 0; i < NR_IRQS; i++) { 49824eeb568SKenji Kaneshige info = &iosapic_intr_info[i]; 49924eeb568SKenji Kaneshige if (info->trigger == trigger && info->polarity == pol && 500f8c087f3SYasuaki Ishimatsu (info->dmode == IOSAPIC_FIXED || 501f8c087f3SYasuaki Ishimatsu info->dmode == IOSAPIC_LOWEST_PRIORITY) && 502f8c087f3SYasuaki Ishimatsu can_request_irq(i, IRQF_SHARED)) { 50324eeb568SKenji Kaneshige if (min_count == -1 || info->count < min_count) { 5044bbdec7aSYasuaki Ishimatsu irq = i; 50524eeb568SKenji Kaneshige min_count = info->count; 50624eeb568SKenji Kaneshige } 50724eeb568SKenji Kaneshige } 50824eeb568SKenji Kaneshige } 5094bbdec7aSYasuaki Ishimatsu return irq; 51024eeb568SKenji Kaneshige } 51124eeb568SKenji Kaneshige 5121da177e4SLinus Torvalds /* 5131da177e4SLinus Torvalds * if the given vector is already owned by other, 5141da177e4SLinus Torvalds * assign a new vector for the other and make the vector available 5151da177e4SLinus Torvalds */ 5161da177e4SLinus Torvalds static void __init 5174bbdec7aSYasuaki Ishimatsu iosapic_reassign_vector (int irq) 5181da177e4SLinus Torvalds { 5194bbdec7aSYasuaki Ishimatsu int new_irq; 5201da177e4SLinus Torvalds 521c4c376f7SKenji Kaneshige if (iosapic_intr_info[irq].count) { 5224bbdec7aSYasuaki Ishimatsu new_irq = create_irq(); 5234bbdec7aSYasuaki Ishimatsu if (new_irq < 0) 524d4ed8084SHarvey Harrison panic("%s: out of interrupt vectors!\n", __func__); 52546cba3dcSSatoru Takeuchi printk(KERN_INFO "Reassigning vector %d to %d\n", 5264bbdec7aSYasuaki Ishimatsu irq_to_vector(irq), irq_to_vector(new_irq)); 5274bbdec7aSYasuaki Ishimatsu memcpy(&iosapic_intr_info[new_irq], &iosapic_intr_info[irq], 5281da177e4SLinus Torvalds sizeof(struct iosapic_intr_info)); 5294bbdec7aSYasuaki Ishimatsu INIT_LIST_HEAD(&iosapic_intr_info[new_irq].rtes); 5304bbdec7aSYasuaki Ishimatsu list_move(iosapic_intr_info[irq].rtes.next, 5314bbdec7aSYasuaki Ishimatsu &iosapic_intr_info[new_irq].rtes); 5324bbdec7aSYasuaki Ishimatsu memset(&iosapic_intr_info[irq], 0, 53346cba3dcSSatoru Takeuchi sizeof(struct iosapic_intr_info)); 5344bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].low32 = IOSAPIC_MASK; 5354bbdec7aSYasuaki Ishimatsu INIT_LIST_HEAD(&iosapic_intr_info[irq].rtes); 5361da177e4SLinus Torvalds } 5371da177e4SLinus Torvalds } 5381da177e4SLinus Torvalds 5394bbdec7aSYasuaki Ishimatsu static inline int irq_is_shared (int irq) 54024eeb568SKenji Kaneshige { 5414bbdec7aSYasuaki Ishimatsu return (iosapic_intr_info[irq].count > 1); 54224eeb568SKenji Kaneshige } 54324eeb568SKenji Kaneshige 54433b39e84SIsaku Yamahata struct irq_chip* 54533b39e84SIsaku Yamahata ia64_native_iosapic_get_irq_chip(unsigned long trigger) 54633b39e84SIsaku Yamahata { 54733b39e84SIsaku Yamahata if (trigger == IOSAPIC_EDGE) 54833b39e84SIsaku Yamahata return &irq_type_iosapic_edge; 54933b39e84SIsaku Yamahata else 55033b39e84SIsaku Yamahata return &irq_type_iosapic_level; 55133b39e84SIsaku Yamahata } 55233b39e84SIsaku Yamahata 55314454a1bSKenji Kaneshige static int 5544bbdec7aSYasuaki Ishimatsu register_intr (unsigned int gsi, int irq, unsigned char delivery, 5551da177e4SLinus Torvalds unsigned long polarity, unsigned long trigger) 5561da177e4SLinus Torvalds { 557dea1078eSThomas Gleixner struct irq_chip *chip, *irq_type; 5581da177e4SLinus Torvalds int index; 55924eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 5601da177e4SLinus Torvalds 5611da177e4SLinus Torvalds index = find_iosapic(gsi); 5621da177e4SLinus Torvalds if (index < 0) { 56346cba3dcSSatoru Takeuchi printk(KERN_WARNING "%s: No IOSAPIC for GSI %u\n", 564d4ed8084SHarvey Harrison __func__, gsi); 56514454a1bSKenji Kaneshige return -ENODEV; 5661da177e4SLinus Torvalds } 5671da177e4SLinus Torvalds 5684bbdec7aSYasuaki Ishimatsu rte = find_rte(irq, gsi); 56924eeb568SKenji Kaneshige if (!rte) { 5704de0a759STony Luck rte = kzalloc(sizeof (*rte), GFP_ATOMIC); 57124eeb568SKenji Kaneshige if (!rte) { 57246cba3dcSSatoru Takeuchi printk(KERN_WARNING "%s: cannot allocate memory\n", 573d4ed8084SHarvey Harrison __func__); 57414454a1bSKenji Kaneshige return -ENOMEM; 57524eeb568SKenji Kaneshige } 57624eeb568SKenji Kaneshige 577c5e3f9e5SYasuaki Ishimatsu rte->iosapic = &iosapic_lists[index]; 578c5e3f9e5SYasuaki Ishimatsu rte->rte_index = gsi - rte->iosapic->gsi_base; 57924eeb568SKenji Kaneshige rte->refcnt++; 5804bbdec7aSYasuaki Ishimatsu list_add_tail(&rte->rte_list, &iosapic_intr_info[irq].rtes); 5814bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].count++; 5820e888adcSKenji Kaneshige iosapic_lists[index].rtes_inuse++; 58324eeb568SKenji Kaneshige } 584e1b30a39SYasuaki Ishimatsu else if (rte->refcnt == NO_REF_RTE) { 5854bbdec7aSYasuaki Ishimatsu struct iosapic_intr_info *info = &iosapic_intr_info[irq]; 586e1b30a39SYasuaki Ishimatsu if (info->count > 0 && 587e1b30a39SYasuaki Ishimatsu (info->trigger != trigger || info->polarity != polarity)){ 58846cba3dcSSatoru Takeuchi printk (KERN_WARNING 58946cba3dcSSatoru Takeuchi "%s: cannot override the interrupt\n", 590d4ed8084SHarvey Harrison __func__); 59114454a1bSKenji Kaneshige return -EINVAL; 59224eeb568SKenji Kaneshige } 593e1b30a39SYasuaki Ishimatsu rte->refcnt++; 594e1b30a39SYasuaki Ishimatsu iosapic_intr_info[irq].count++; 595e1b30a39SYasuaki Ishimatsu iosapic_lists[index].rtes_inuse++; 59624eeb568SKenji Kaneshige } 59724eeb568SKenji Kaneshige 5984bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].polarity = polarity; 5994bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].dmode = delivery; 6004bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].trigger = trigger; 6011da177e4SLinus Torvalds 60233b39e84SIsaku Yamahata irq_type = iosapic_get_irq_chip(trigger); 6031da177e4SLinus Torvalds 604dea1078eSThomas Gleixner chip = irq_get_chip(irq); 605dea1078eSThomas Gleixner if (irq_type != NULL && chip != irq_type) { 606dea1078eSThomas Gleixner if (chip != &no_irq_chip) 60746cba3dcSSatoru Takeuchi printk(KERN_WARNING 60846cba3dcSSatoru Takeuchi "%s: changing vector %d from %s to %s\n", 609d4ed8084SHarvey Harrison __func__, irq_to_vector(irq), 610dea1078eSThomas Gleixner chip->name, irq_type->name); 611dea1078eSThomas Gleixner chip = irq_type; 6121da177e4SLinus Torvalds } 613dea1078eSThomas Gleixner __irq_set_chip_handler_name_locked(irq, chip, trigger == IOSAPIC_EDGE ? 614dea1078eSThomas Gleixner handle_edge_irq : handle_level_irq, 615dea1078eSThomas Gleixner NULL); 61614454a1bSKenji Kaneshige return 0; 6171da177e4SLinus Torvalds } 6181da177e4SLinus Torvalds 6191da177e4SLinus Torvalds static unsigned int 6204bbdec7aSYasuaki Ishimatsu get_target_cpu (unsigned int gsi, int irq) 6211da177e4SLinus Torvalds { 6221da177e4SLinus Torvalds #ifdef CONFIG_SMP 6231da177e4SLinus Torvalds static int cpu = -1; 624ff741906SAshok Raj extern int cpe_vector; 6254994be1bSYasuaki Ishimatsu cpumask_t domain = irq_to_domain(irq); 6261da177e4SLinus Torvalds 6271da177e4SLinus Torvalds /* 62824eeb568SKenji Kaneshige * In case of vector shared by multiple RTEs, all RTEs that 62924eeb568SKenji Kaneshige * share the vector need to use the same destination CPU. 63024eeb568SKenji Kaneshige */ 631c4c376f7SKenji Kaneshige if (iosapic_intr_info[irq].count) 6324bbdec7aSYasuaki Ishimatsu return iosapic_intr_info[irq].dest; 63324eeb568SKenji Kaneshige 63424eeb568SKenji Kaneshige /* 6351da177e4SLinus Torvalds * If the platform supports redirection via XTP, let it 6361da177e4SLinus Torvalds * distribute interrupts. 6371da177e4SLinus Torvalds */ 6381da177e4SLinus Torvalds if (smp_int_redirect & SMP_IRQ_REDIRECTION) 6391da177e4SLinus Torvalds return cpu_physical_id(smp_processor_id()); 6401da177e4SLinus Torvalds 6411da177e4SLinus Torvalds /* 6421da177e4SLinus Torvalds * Some interrupts (ACPI SCI, for instance) are registered 6431da177e4SLinus Torvalds * before the BSP is marked as online. 6441da177e4SLinus Torvalds */ 6451da177e4SLinus Torvalds if (!cpu_online(smp_processor_id())) 6461da177e4SLinus Torvalds return cpu_physical_id(smp_processor_id()); 6471da177e4SLinus Torvalds 648ff741906SAshok Raj #ifdef CONFIG_ACPI 6494bbdec7aSYasuaki Ishimatsu if (cpe_vector > 0 && irq_to_vector(irq) == IA64_CPEP_VECTOR) 650ff741906SAshok Raj return get_cpei_target_cpu(); 651ff741906SAshok Raj #endif 652ff741906SAshok Raj 6531da177e4SLinus Torvalds #ifdef CONFIG_NUMA 6541da177e4SLinus Torvalds { 6551da177e4SLinus Torvalds int num_cpus, cpu_index, iosapic_index, numa_cpu, i = 0; 656fbb776c3SRusty Russell const struct cpumask *cpu_mask; 6571da177e4SLinus Torvalds 6581da177e4SLinus Torvalds iosapic_index = find_iosapic(gsi); 6591da177e4SLinus Torvalds if (iosapic_index < 0 || 6601da177e4SLinus Torvalds iosapic_lists[iosapic_index].node == MAX_NUMNODES) 6611da177e4SLinus Torvalds goto skip_numa_setup; 6621da177e4SLinus Torvalds 663fbb776c3SRusty Russell cpu_mask = cpumask_of_node(iosapic_lists[iosapic_index].node); 664fbb776c3SRusty Russell num_cpus = 0; 665fbb776c3SRusty Russell for_each_cpu_and(numa_cpu, cpu_mask, &domain) { 666fbb776c3SRusty Russell if (cpu_online(numa_cpu)) 667fbb776c3SRusty Russell num_cpus++; 6681da177e4SLinus Torvalds } 6691da177e4SLinus Torvalds 6701da177e4SLinus Torvalds if (!num_cpus) 6711da177e4SLinus Torvalds goto skip_numa_setup; 6721da177e4SLinus Torvalds 6734bbdec7aSYasuaki Ishimatsu /* Use irq assignment to distribute across cpus in node */ 6744bbdec7aSYasuaki Ishimatsu cpu_index = irq % num_cpus; 6751da177e4SLinus Torvalds 676fbb776c3SRusty Russell for_each_cpu_and(numa_cpu, cpu_mask, &domain) 677fbb776c3SRusty Russell if (cpu_online(numa_cpu) && i++ >= cpu_index) 678fbb776c3SRusty Russell break; 6791da177e4SLinus Torvalds 680fbb776c3SRusty Russell if (numa_cpu < nr_cpu_ids) 6811da177e4SLinus Torvalds return cpu_physical_id(numa_cpu); 6821da177e4SLinus Torvalds } 6831da177e4SLinus Torvalds skip_numa_setup: 6841da177e4SLinus Torvalds #endif 6851da177e4SLinus Torvalds /* 6861da177e4SLinus Torvalds * Otherwise, round-robin interrupt vectors across all the 6871da177e4SLinus Torvalds * processors. (It'd be nice if we could be smarter in the 6881da177e4SLinus Torvalds * case of NUMA.) 6891da177e4SLinus Torvalds */ 6901da177e4SLinus Torvalds do { 691fbb776c3SRusty Russell if (++cpu >= nr_cpu_ids) 6921da177e4SLinus Torvalds cpu = 0; 6935d2068daSRusty Russell } while (!cpu_online(cpu) || !cpumask_test_cpu(cpu, &domain)); 6941da177e4SLinus Torvalds 6951da177e4SLinus Torvalds return cpu_physical_id(cpu); 69646cba3dcSSatoru Takeuchi #else /* CONFIG_SMP */ 6971da177e4SLinus Torvalds return cpu_physical_id(smp_processor_id()); 6981da177e4SLinus Torvalds #endif 6991da177e4SLinus Torvalds } 7001da177e4SLinus Torvalds 701c9d059deSKenji Kaneshige static inline unsigned char choose_dmode(void) 702c9d059deSKenji Kaneshige { 703c9d059deSKenji Kaneshige #ifdef CONFIG_SMP 704c9d059deSKenji Kaneshige if (smp_int_redirect & SMP_IRQ_REDIRECTION) 705c9d059deSKenji Kaneshige return IOSAPIC_LOWEST_PRIORITY; 706c9d059deSKenji Kaneshige #endif 707c9d059deSKenji Kaneshige return IOSAPIC_FIXED; 708c9d059deSKenji Kaneshige } 709c9d059deSKenji Kaneshige 7101da177e4SLinus Torvalds /* 7111da177e4SLinus Torvalds * ACPI can describe IOSAPIC interrupts via static tables and namespace 7121da177e4SLinus Torvalds * methods. This provides an interface to register those interrupts and 7131da177e4SLinus Torvalds * program the IOSAPIC RTE. 7141da177e4SLinus Torvalds */ 7151da177e4SLinus Torvalds int 7161da177e4SLinus Torvalds iosapic_register_intr (unsigned int gsi, 7171da177e4SLinus Torvalds unsigned long polarity, unsigned long trigger) 7181da177e4SLinus Torvalds { 7194bbdec7aSYasuaki Ishimatsu int irq, mask = 1, err; 7201da177e4SLinus Torvalds unsigned int dest; 7211da177e4SLinus Torvalds unsigned long flags; 72224eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 72324eeb568SKenji Kaneshige u32 low32; 724c9d059deSKenji Kaneshige unsigned char dmode; 725dea1078eSThomas Gleixner struct irq_desc *desc; 72640598cbeSYasuaki Ishimatsu 7271da177e4SLinus Torvalds /* 7281da177e4SLinus Torvalds * If this GSI has already been registered (i.e., it's a 7291da177e4SLinus Torvalds * shared interrupt, or we lost a race to register it), 7301da177e4SLinus Torvalds * don't touch the RTE. 7311da177e4SLinus Torvalds */ 7321da177e4SLinus Torvalds spin_lock_irqsave(&iosapic_lock, flags); 7334bbdec7aSYasuaki Ishimatsu irq = __gsi_to_irq(gsi); 7344bbdec7aSYasuaki Ishimatsu if (irq > 0) { 7354bbdec7aSYasuaki Ishimatsu rte = find_rte(irq, gsi); 736e1b30a39SYasuaki Ishimatsu if(iosapic_intr_info[irq].count == 0) { 737e1b30a39SYasuaki Ishimatsu assign_irq_vector(irq); 7384debd723SThomas Gleixner irq_init_desc(irq); 739e1b30a39SYasuaki Ishimatsu } else if (rte->refcnt != NO_REF_RTE) { 74024eeb568SKenji Kaneshige rte->refcnt++; 74140598cbeSYasuaki Ishimatsu goto unlock_iosapic_lock; 7421da177e4SLinus Torvalds } 743e1b30a39SYasuaki Ishimatsu } else 744e1b30a39SYasuaki Ishimatsu irq = create_irq(); 7451da177e4SLinus Torvalds 74624eeb568SKenji Kaneshige /* If vector is running out, we try to find a sharable vector */ 747eb21ab24SYasuaki Ishimatsu if (irq < 0) { 7484bbdec7aSYasuaki Ishimatsu irq = iosapic_find_sharable_irq(trigger, polarity); 7494bbdec7aSYasuaki Ishimatsu if (irq < 0) 75040598cbeSYasuaki Ishimatsu goto unlock_iosapic_lock; 7514bbdec7aSYasuaki Ishimatsu } 75224eeb568SKenji Kaneshige 753dea1078eSThomas Gleixner desc = irq_to_desc(irq); 754dea1078eSThomas Gleixner raw_spin_lock(&desc->lock); 7554bbdec7aSYasuaki Ishimatsu dest = get_target_cpu(gsi, irq); 756c9d059deSKenji Kaneshige dmode = choose_dmode(); 757c9d059deSKenji Kaneshige err = register_intr(gsi, irq, dmode, polarity, trigger); 75814454a1bSKenji Kaneshige if (err < 0) { 759dea1078eSThomas Gleixner raw_spin_unlock(&desc->lock); 7604bbdec7aSYasuaki Ishimatsu irq = err; 761224685c0SKenji Kaneshige goto unlock_iosapic_lock; 76214454a1bSKenji Kaneshige } 7631da177e4SLinus Torvalds 76424eeb568SKenji Kaneshige /* 765e3a8f7b8SYasuaki Ishimatsu * If the vector is shared and already unmasked for other 766e3a8f7b8SYasuaki Ishimatsu * interrupt sources, don't mask it. 76724eeb568SKenji Kaneshige */ 7684bbdec7aSYasuaki Ishimatsu low32 = iosapic_intr_info[irq].low32; 7694bbdec7aSYasuaki Ishimatsu if (irq_is_shared(irq) && !(low32 & IOSAPIC_MASK)) 77024eeb568SKenji Kaneshige mask = 0; 7714bbdec7aSYasuaki Ishimatsu set_rte(gsi, irq, dest, mask); 7721da177e4SLinus Torvalds 7731da177e4SLinus Torvalds printk(KERN_INFO "GSI %u (%s, %s) -> CPU %d (0x%04x) vector %d\n", 7741da177e4SLinus Torvalds gsi, (trigger == IOSAPIC_EDGE ? "edge" : "level"), 7751da177e4SLinus Torvalds (polarity == IOSAPIC_POL_HIGH ? "high" : "low"), 7764bbdec7aSYasuaki Ishimatsu cpu_logical_id(dest), dest, irq_to_vector(irq)); 777224685c0SKenji Kaneshige 778dea1078eSThomas Gleixner raw_spin_unlock(&desc->lock); 77940598cbeSYasuaki Ishimatsu unlock_iosapic_lock: 78040598cbeSYasuaki Ishimatsu spin_unlock_irqrestore(&iosapic_lock, flags); 7814bbdec7aSYasuaki Ishimatsu return irq; 7821da177e4SLinus Torvalds } 7831da177e4SLinus Torvalds 7841da177e4SLinus Torvalds void 7851da177e4SLinus Torvalds iosapic_unregister_intr (unsigned int gsi) 7861da177e4SLinus Torvalds { 7871da177e4SLinus Torvalds unsigned long flags; 7884bbdec7aSYasuaki Ishimatsu int irq, index; 78924eeb568SKenji Kaneshige u32 low32; 7901da177e4SLinus Torvalds unsigned long trigger, polarity; 79124eeb568SKenji Kaneshige unsigned int dest; 79224eeb568SKenji Kaneshige struct iosapic_rte_info *rte; 7931da177e4SLinus Torvalds 7941da177e4SLinus Torvalds /* 7951da177e4SLinus Torvalds * If the irq associated with the gsi is not found, 7961da177e4SLinus Torvalds * iosapic_unregister_intr() is unbalanced. We need to check 7971da177e4SLinus Torvalds * this again after getting locks. 7981da177e4SLinus Torvalds */ 7991da177e4SLinus Torvalds irq = gsi_to_irq(gsi); 8001da177e4SLinus Torvalds if (irq < 0) { 80146cba3dcSSatoru Takeuchi printk(KERN_ERR "iosapic_unregister_intr(%u) unbalanced\n", 80246cba3dcSSatoru Takeuchi gsi); 8031da177e4SLinus Torvalds WARN_ON(1); 8041da177e4SLinus Torvalds return; 8051da177e4SLinus Torvalds } 8061da177e4SLinus Torvalds 80740598cbeSYasuaki Ishimatsu spin_lock_irqsave(&iosapic_lock, flags); 8084bbdec7aSYasuaki Ishimatsu if ((rte = find_rte(irq, gsi)) == NULL) { 809e3a8f7b8SYasuaki Ishimatsu printk(KERN_ERR "iosapic_unregister_intr(%u) unbalanced\n", 81046cba3dcSSatoru Takeuchi gsi); 8111da177e4SLinus Torvalds WARN_ON(1); 81224eeb568SKenji Kaneshige goto out; 8131da177e4SLinus Torvalds } 8141da177e4SLinus Torvalds 81524eeb568SKenji Kaneshige if (--rte->refcnt > 0) 81624eeb568SKenji Kaneshige goto out; 8171da177e4SLinus Torvalds 818e1b30a39SYasuaki Ishimatsu rte->refcnt = NO_REF_RTE; 81940598cbeSYasuaki Ishimatsu 82024eeb568SKenji Kaneshige /* Mask the interrupt */ 8214bbdec7aSYasuaki Ishimatsu low32 = iosapic_intr_info[irq].low32 | IOSAPIC_MASK; 822c1726d6fSYasuaki Ishimatsu iosapic_write(rte->iosapic, IOSAPIC_RTE_LOW(rte->rte_index), low32); 8231da177e4SLinus Torvalds 8244bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].count--; 8250e888adcSKenji Kaneshige index = find_iosapic(gsi); 8260e888adcSKenji Kaneshige iosapic_lists[index].rtes_inuse--; 8270e888adcSKenji Kaneshige WARN_ON(iosapic_lists[index].rtes_inuse < 0); 8281da177e4SLinus Torvalds 8294bbdec7aSYasuaki Ishimatsu trigger = iosapic_intr_info[irq].trigger; 8304bbdec7aSYasuaki Ishimatsu polarity = iosapic_intr_info[irq].polarity; 8314bbdec7aSYasuaki Ishimatsu dest = iosapic_intr_info[irq].dest; 83246cba3dcSSatoru Takeuchi printk(KERN_INFO 833e3a8f7b8SYasuaki Ishimatsu "GSI %u (%s, %s) -> CPU %d (0x%04x) vector %d unregistered\n", 83424eeb568SKenji Kaneshige gsi, (trigger == IOSAPIC_EDGE ? "edge" : "level"), 83524eeb568SKenji Kaneshige (polarity == IOSAPIC_POL_HIGH ? "high" : "low"), 8364bbdec7aSYasuaki Ishimatsu cpu_logical_id(dest), dest, irq_to_vector(irq)); 8371da177e4SLinus Torvalds 838e1b30a39SYasuaki Ishimatsu if (iosapic_intr_info[irq].count == 0) { 839451fe00cSAlex Williamson #ifdef CONFIG_SMP 840451fe00cSAlex Williamson /* Clear affinity */ 841dea1078eSThomas Gleixner cpumask_setall(irq_get_irq_data(irq)->affinity); 842451fe00cSAlex Williamson #endif 84324eeb568SKenji Kaneshige /* Clear the interrupt information */ 844e1b30a39SYasuaki Ishimatsu iosapic_intr_info[irq].dest = 0; 845e1b30a39SYasuaki Ishimatsu iosapic_intr_info[irq].dmode = 0; 846e1b30a39SYasuaki Ishimatsu iosapic_intr_info[irq].polarity = 0; 847e1b30a39SYasuaki Ishimatsu iosapic_intr_info[irq].trigger = 0; 8484bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].low32 |= IOSAPIC_MASK; 84924eeb568SKenji Kaneshige 850e1b30a39SYasuaki Ishimatsu /* Destroy and reserve IRQ */ 851e1b30a39SYasuaki Ishimatsu destroy_and_reserve_irq(irq); 85224eeb568SKenji Kaneshige } 85324eeb568SKenji Kaneshige out: 85440598cbeSYasuaki Ishimatsu spin_unlock_irqrestore(&iosapic_lock, flags); 8551da177e4SLinus Torvalds } 8561da177e4SLinus Torvalds 8571da177e4SLinus Torvalds /* 8581da177e4SLinus Torvalds * ACPI calls this when it finds an entry for a platform interrupt. 8591da177e4SLinus Torvalds */ 8601da177e4SLinus Torvalds int __init 8611da177e4SLinus Torvalds iosapic_register_platform_intr (u32 int_type, unsigned int gsi, 8621da177e4SLinus Torvalds int iosapic_vector, u16 eid, u16 id, 8631da177e4SLinus Torvalds unsigned long polarity, unsigned long trigger) 8641da177e4SLinus Torvalds { 8651da177e4SLinus Torvalds static const char * const name[] = {"unknown", "PMI", "INIT", "CPEI"}; 8661da177e4SLinus Torvalds unsigned char delivery; 867eb21ab24SYasuaki Ishimatsu int irq, vector, mask = 0; 8681da177e4SLinus Torvalds unsigned int dest = ((id << 8) | eid) & 0xffff; 8691da177e4SLinus Torvalds 8701da177e4SLinus Torvalds switch (int_type) { 8711da177e4SLinus Torvalds case ACPI_INTERRUPT_PMI: 872e1b30a39SYasuaki Ishimatsu irq = vector = iosapic_vector; 8734994be1bSYasuaki Ishimatsu bind_irq_vector(irq, vector, CPU_MASK_ALL); 8741da177e4SLinus Torvalds /* 8751da177e4SLinus Torvalds * since PMI vector is alloc'd by FW(ACPI) not by kernel, 8761da177e4SLinus Torvalds * we need to make sure the vector is available 8771da177e4SLinus Torvalds */ 8784bbdec7aSYasuaki Ishimatsu iosapic_reassign_vector(irq); 8791da177e4SLinus Torvalds delivery = IOSAPIC_PMI; 8801da177e4SLinus Torvalds break; 8811da177e4SLinus Torvalds case ACPI_INTERRUPT_INIT: 882eb21ab24SYasuaki Ishimatsu irq = create_irq(); 883eb21ab24SYasuaki Ishimatsu if (irq < 0) 884d4ed8084SHarvey Harrison panic("%s: out of interrupt vectors!\n", __func__); 885eb21ab24SYasuaki Ishimatsu vector = irq_to_vector(irq); 8861da177e4SLinus Torvalds delivery = IOSAPIC_INIT; 8871da177e4SLinus Torvalds break; 8881da177e4SLinus Torvalds case ACPI_INTERRUPT_CPEI: 889e1b30a39SYasuaki Ishimatsu irq = vector = IA64_CPE_VECTOR; 8904994be1bSYasuaki Ishimatsu BUG_ON(bind_irq_vector(irq, vector, CPU_MASK_ALL)); 891aa0ebec9SKenji Kaneshige delivery = IOSAPIC_FIXED; 8921da177e4SLinus Torvalds mask = 1; 8931da177e4SLinus Torvalds break; 8941da177e4SLinus Torvalds default: 895d4ed8084SHarvey Harrison printk(KERN_ERR "%s: invalid int type 0x%x\n", __func__, 89646cba3dcSSatoru Takeuchi int_type); 8971da177e4SLinus Torvalds return -1; 8981da177e4SLinus Torvalds } 8991da177e4SLinus Torvalds 9004bbdec7aSYasuaki Ishimatsu register_intr(gsi, irq, delivery, polarity, trigger); 9011da177e4SLinus Torvalds 90246cba3dcSSatoru Takeuchi printk(KERN_INFO 90346cba3dcSSatoru Takeuchi "PLATFORM int %s (0x%x): GSI %u (%s, %s) -> CPU %d (0x%04x)" 90446cba3dcSSatoru Takeuchi " vector %d\n", 9051da177e4SLinus Torvalds int_type < ARRAY_SIZE(name) ? name[int_type] : "unknown", 9061da177e4SLinus Torvalds int_type, gsi, (trigger == IOSAPIC_EDGE ? "edge" : "level"), 9071da177e4SLinus Torvalds (polarity == IOSAPIC_POL_HIGH ? "high" : "low"), 9081da177e4SLinus Torvalds cpu_logical_id(dest), dest, vector); 9091da177e4SLinus Torvalds 9104bbdec7aSYasuaki Ishimatsu set_rte(gsi, irq, dest, mask); 9111da177e4SLinus Torvalds return vector; 9121da177e4SLinus Torvalds } 9131da177e4SLinus Torvalds 9141da177e4SLinus Torvalds /* 9151da177e4SLinus Torvalds * ACPI calls this when it finds an entry for a legacy ISA IRQ override. 9161da177e4SLinus Torvalds */ 9175b5e76e9SGreg Kroah-Hartman void iosapic_override_isa_irq(unsigned int isa_irq, unsigned int gsi, 9185b5e76e9SGreg Kroah-Hartman unsigned long polarity, unsigned long trigger) 9191da177e4SLinus Torvalds { 9204bbdec7aSYasuaki Ishimatsu int vector, irq; 9211da177e4SLinus Torvalds unsigned int dest = cpu_physical_id(smp_processor_id()); 922c9d059deSKenji Kaneshige unsigned char dmode; 9231da177e4SLinus Torvalds 924e1b30a39SYasuaki Ishimatsu irq = vector = isa_irq_to_vector(isa_irq); 9254994be1bSYasuaki Ishimatsu BUG_ON(bind_irq_vector(irq, vector, CPU_MASK_ALL)); 926c9d059deSKenji Kaneshige dmode = choose_dmode(); 927c9d059deSKenji Kaneshige register_intr(gsi, irq, dmode, polarity, trigger); 9281da177e4SLinus Torvalds 9291da177e4SLinus Torvalds DBG("ISA: IRQ %u -> GSI %u (%s,%s) -> CPU %d (0x%04x) vector %d\n", 9301da177e4SLinus Torvalds isa_irq, gsi, trigger == IOSAPIC_EDGE ? "edge" : "level", 9311da177e4SLinus Torvalds polarity == IOSAPIC_POL_HIGH ? "high" : "low", 9321da177e4SLinus Torvalds cpu_logical_id(dest), dest, vector); 9331da177e4SLinus Torvalds 9344bbdec7aSYasuaki Ishimatsu set_rte(gsi, irq, dest, 1); 9351da177e4SLinus Torvalds } 9361da177e4SLinus Torvalds 9371da177e4SLinus Torvalds void __init 93833b39e84SIsaku Yamahata ia64_native_iosapic_pcat_compat_init(void) 93933b39e84SIsaku Yamahata { 94033b39e84SIsaku Yamahata if (pcat_compat) { 94133b39e84SIsaku Yamahata /* 94233b39e84SIsaku Yamahata * Disable the compatibility mode interrupts (8259 style), 94333b39e84SIsaku Yamahata * needs IN/OUT support enabled. 94433b39e84SIsaku Yamahata */ 94533b39e84SIsaku Yamahata printk(KERN_INFO 94633b39e84SIsaku Yamahata "%s: Disabling PC-AT compatible 8259 interrupts\n", 94733b39e84SIsaku Yamahata __func__); 94833b39e84SIsaku Yamahata outb(0xff, 0xA1); 94933b39e84SIsaku Yamahata outb(0xff, 0x21); 95033b39e84SIsaku Yamahata } 95133b39e84SIsaku Yamahata } 95233b39e84SIsaku Yamahata 95333b39e84SIsaku Yamahata void __init 9541da177e4SLinus Torvalds iosapic_system_init (int system_pcat_compat) 9551da177e4SLinus Torvalds { 9564bbdec7aSYasuaki Ishimatsu int irq; 9571da177e4SLinus Torvalds 9584bbdec7aSYasuaki Ishimatsu for (irq = 0; irq < NR_IRQS; ++irq) { 9594bbdec7aSYasuaki Ishimatsu iosapic_intr_info[irq].low32 = IOSAPIC_MASK; 96046cba3dcSSatoru Takeuchi /* mark as unused */ 9614bbdec7aSYasuaki Ishimatsu INIT_LIST_HEAD(&iosapic_intr_info[irq].rtes); 962e1b30a39SYasuaki Ishimatsu 963e1b30a39SYasuaki Ishimatsu iosapic_intr_info[irq].count = 0; 96424eeb568SKenji Kaneshige } 9651da177e4SLinus Torvalds 9661da177e4SLinus Torvalds pcat_compat = system_pcat_compat; 96733b39e84SIsaku Yamahata if (pcat_compat) 96833b39e84SIsaku Yamahata iosapic_pcat_compat_init(); 9691da177e4SLinus Torvalds } 9701da177e4SLinus Torvalds 9710e888adcSKenji Kaneshige static inline int 9720e888adcSKenji Kaneshige iosapic_alloc (void) 9730e888adcSKenji Kaneshige { 9740e888adcSKenji Kaneshige int index; 9750e888adcSKenji Kaneshige 9760e888adcSKenji Kaneshige for (index = 0; index < NR_IOSAPICS; index++) 9770e888adcSKenji Kaneshige if (!iosapic_lists[index].addr) 9780e888adcSKenji Kaneshige return index; 9790e888adcSKenji Kaneshige 980d4ed8084SHarvey Harrison printk(KERN_WARNING "%s: failed to allocate iosapic\n", __func__); 9810e888adcSKenji Kaneshige return -1; 9820e888adcSKenji Kaneshige } 9830e888adcSKenji Kaneshige 9840e888adcSKenji Kaneshige static inline void 9850e888adcSKenji Kaneshige iosapic_free (int index) 9860e888adcSKenji Kaneshige { 9870e888adcSKenji Kaneshige memset(&iosapic_lists[index], 0, sizeof(iosapic_lists[0])); 9880e888adcSKenji Kaneshige } 9890e888adcSKenji Kaneshige 9900e888adcSKenji Kaneshige static inline int 9910e888adcSKenji Kaneshige iosapic_check_gsi_range (unsigned int gsi_base, unsigned int ver) 9920e888adcSKenji Kaneshige { 9930e888adcSKenji Kaneshige int index; 9940e888adcSKenji Kaneshige unsigned int gsi_end, base, end; 9950e888adcSKenji Kaneshige 9960e888adcSKenji Kaneshige /* check gsi range */ 9970e888adcSKenji Kaneshige gsi_end = gsi_base + ((ver >> 16) & 0xff); 9980e888adcSKenji Kaneshige for (index = 0; index < NR_IOSAPICS; index++) { 9990e888adcSKenji Kaneshige if (!iosapic_lists[index].addr) 10000e888adcSKenji Kaneshige continue; 10010e888adcSKenji Kaneshige 10020e888adcSKenji Kaneshige base = iosapic_lists[index].gsi_base; 10030e888adcSKenji Kaneshige end = base + iosapic_lists[index].num_rte - 1; 10040e888adcSKenji Kaneshige 1005e6d1ba5cSSatoru Takeuchi if (gsi_end < base || end < gsi_base) 10060e888adcSKenji Kaneshige continue; /* OK */ 10070e888adcSKenji Kaneshige 10080e888adcSKenji Kaneshige return -EBUSY; 10090e888adcSKenji Kaneshige } 10100e888adcSKenji Kaneshige return 0; 10110e888adcSKenji Kaneshige } 10120e888adcSKenji Kaneshige 1013ffa90955SHanjun Guo static int 1014ffa90955SHanjun Guo iosapic_delete_rte(unsigned int irq, unsigned int gsi) 1015ffa90955SHanjun Guo { 1016ffa90955SHanjun Guo struct iosapic_rte_info *rte, *temp; 1017ffa90955SHanjun Guo 1018ffa90955SHanjun Guo list_for_each_entry_safe(rte, temp, &iosapic_intr_info[irq].rtes, 1019ffa90955SHanjun Guo rte_list) { 1020ffa90955SHanjun Guo if (rte->iosapic->gsi_base + rte->rte_index == gsi) { 1021ffa90955SHanjun Guo if (rte->refcnt) 1022ffa90955SHanjun Guo return -EBUSY; 1023ffa90955SHanjun Guo 1024ffa90955SHanjun Guo list_del(&rte->rte_list); 1025ffa90955SHanjun Guo kfree(rte); 1026ffa90955SHanjun Guo return 0; 1027ffa90955SHanjun Guo } 1028ffa90955SHanjun Guo } 1029ffa90955SHanjun Guo 1030ffa90955SHanjun Guo return -EINVAL; 1031ffa90955SHanjun Guo } 1032ffa90955SHanjun Guo 10335b5e76e9SGreg Kroah-Hartman int iosapic_init(unsigned long phys_addr, unsigned int gsi_base) 10341da177e4SLinus Torvalds { 10350e888adcSKenji Kaneshige int num_rte, err, index; 10361da177e4SLinus Torvalds unsigned int isa_irq, ver; 10371da177e4SLinus Torvalds char __iomem *addr; 10380e888adcSKenji Kaneshige unsigned long flags; 10391da177e4SLinus Torvalds 10400e888adcSKenji Kaneshige spin_lock_irqsave(&iosapic_lock, flags); 1041c1726d6fSYasuaki Ishimatsu index = find_iosapic(gsi_base); 1042c1726d6fSYasuaki Ishimatsu if (index >= 0) { 1043c1726d6fSYasuaki Ishimatsu spin_unlock_irqrestore(&iosapic_lock, flags); 1044c1726d6fSYasuaki Ishimatsu return -EBUSY; 1045c1726d6fSYasuaki Ishimatsu } 1046c1726d6fSYasuaki Ishimatsu 10471da177e4SLinus Torvalds addr = ioremap(phys_addr, 0); 1048e7369e01SRoel Kluin if (addr == NULL) { 1049e7369e01SRoel Kluin spin_unlock_irqrestore(&iosapic_lock, flags); 1050e7369e01SRoel Kluin return -ENOMEM; 1051e7369e01SRoel Kluin } 10521da177e4SLinus Torvalds ver = iosapic_version(addr); 10530e888adcSKenji Kaneshige if ((err = iosapic_check_gsi_range(gsi_base, ver))) { 10540e888adcSKenji Kaneshige iounmap(addr); 10550e888adcSKenji Kaneshige spin_unlock_irqrestore(&iosapic_lock, flags); 10560e888adcSKenji Kaneshige return err; 10570e888adcSKenji Kaneshige } 10580e888adcSKenji Kaneshige 10591da177e4SLinus Torvalds /* 1060e3a8f7b8SYasuaki Ishimatsu * The MAX_REDIR register holds the highest input pin number 1061e3a8f7b8SYasuaki Ishimatsu * (starting from 0). We add 1 so that we can use it for 1062e3a8f7b8SYasuaki Ishimatsu * number of pins (= RTEs) 10631da177e4SLinus Torvalds */ 10641da177e4SLinus Torvalds num_rte = ((ver >> 16) & 0xff) + 1; 10651da177e4SLinus Torvalds 10660e888adcSKenji Kaneshige index = iosapic_alloc(); 10670e888adcSKenji Kaneshige iosapic_lists[index].addr = addr; 10680e888adcSKenji Kaneshige iosapic_lists[index].gsi_base = gsi_base; 10690e888adcSKenji Kaneshige iosapic_lists[index].num_rte = num_rte; 10701da177e4SLinus Torvalds #ifdef CONFIG_NUMA 10710e888adcSKenji Kaneshige iosapic_lists[index].node = MAX_NUMNODES; 10721da177e4SLinus Torvalds #endif 1073c1726d6fSYasuaki Ishimatsu spin_lock_init(&iosapic_lists[index].lock); 10740e888adcSKenji Kaneshige spin_unlock_irqrestore(&iosapic_lock, flags); 10751da177e4SLinus Torvalds 10761da177e4SLinus Torvalds if ((gsi_base == 0) && pcat_compat) { 10771da177e4SLinus Torvalds /* 107846cba3dcSSatoru Takeuchi * Map the legacy ISA devices into the IOSAPIC data. Some of 107946cba3dcSSatoru Takeuchi * these may get reprogrammed later on with data from the ACPI 108046cba3dcSSatoru Takeuchi * Interrupt Source Override table. 10811da177e4SLinus Torvalds */ 10821da177e4SLinus Torvalds for (isa_irq = 0; isa_irq < 16; ++isa_irq) 108346cba3dcSSatoru Takeuchi iosapic_override_isa_irq(isa_irq, isa_irq, 108446cba3dcSSatoru Takeuchi IOSAPIC_POL_HIGH, 108546cba3dcSSatoru Takeuchi IOSAPIC_EDGE); 10861da177e4SLinus Torvalds } 10870e888adcSKenji Kaneshige return 0; 10881da177e4SLinus Torvalds } 10891da177e4SLinus Torvalds 10905b5e76e9SGreg Kroah-Hartman int iosapic_remove(unsigned int gsi_base) 10910e888adcSKenji Kaneshige { 1092ffa90955SHanjun Guo int i, irq, index, err = 0; 10930e888adcSKenji Kaneshige unsigned long flags; 10940e888adcSKenji Kaneshige 10950e888adcSKenji Kaneshige spin_lock_irqsave(&iosapic_lock, flags); 10960e888adcSKenji Kaneshige index = find_iosapic(gsi_base); 10970e888adcSKenji Kaneshige if (index < 0) { 10980e888adcSKenji Kaneshige printk(KERN_WARNING "%s: No IOSAPIC for GSI base %u\n", 1099d4ed8084SHarvey Harrison __func__, gsi_base); 11000e888adcSKenji Kaneshige goto out; 11010e888adcSKenji Kaneshige } 11020e888adcSKenji Kaneshige 11030e888adcSKenji Kaneshige if (iosapic_lists[index].rtes_inuse) { 11040e888adcSKenji Kaneshige err = -EBUSY; 1105e3a8f7b8SYasuaki Ishimatsu printk(KERN_WARNING "%s: IOSAPIC for GSI base %u is busy\n", 1106d4ed8084SHarvey Harrison __func__, gsi_base); 11070e888adcSKenji Kaneshige goto out; 11080e888adcSKenji Kaneshige } 11090e888adcSKenji Kaneshige 1110ffa90955SHanjun Guo for (i = gsi_base; i < gsi_base + iosapic_lists[index].num_rte; i++) { 1111ffa90955SHanjun Guo irq = __gsi_to_irq(i); 1112ffa90955SHanjun Guo if (irq < 0) 1113ffa90955SHanjun Guo continue; 1114ffa90955SHanjun Guo 1115ffa90955SHanjun Guo err = iosapic_delete_rte(irq, i); 1116ffa90955SHanjun Guo if (err) 1117ffa90955SHanjun Guo goto out; 1118ffa90955SHanjun Guo } 1119ffa90955SHanjun Guo 11200e888adcSKenji Kaneshige iounmap(iosapic_lists[index].addr); 11210e888adcSKenji Kaneshige iosapic_free(index); 11220e888adcSKenji Kaneshige out: 11230e888adcSKenji Kaneshige spin_unlock_irqrestore(&iosapic_lock, flags); 11240e888adcSKenji Kaneshige return err; 11250e888adcSKenji Kaneshige } 11260e888adcSKenji Kaneshige 11271da177e4SLinus Torvalds #ifdef CONFIG_NUMA 11285b5e76e9SGreg Kroah-Hartman void map_iosapic_to_node(unsigned int gsi_base, int node) 11291da177e4SLinus Torvalds { 11301da177e4SLinus Torvalds int index; 11311da177e4SLinus Torvalds 11321da177e4SLinus Torvalds index = find_iosapic(gsi_base); 11331da177e4SLinus Torvalds if (index < 0) { 11341da177e4SLinus Torvalds printk(KERN_WARNING "%s: No IOSAPIC for GSI %u\n", 1135d4ed8084SHarvey Harrison __func__, gsi_base); 11361da177e4SLinus Torvalds return; 11371da177e4SLinus Torvalds } 11381da177e4SLinus Torvalds iosapic_lists[index].node = node; 11391da177e4SLinus Torvalds return; 11401da177e4SLinus Torvalds } 11411da177e4SLinus Torvalds #endif 1142