1Hexagon is Qualcomm's very long instruction word (VLIW) digital signal 2processor(DSP). We also support Hexagon Vector eXtensions (HVX). HVX 3is a wide vector coprocessor designed for high performance computer vision, 4image processing, machine learning, and other workloads. 5 6The following versions of the Hexagon core are supported 7 Scalar core: v67 8 https://developer.qualcomm.com/downloads/qualcomm-hexagon-v67-programmer-s-reference-manual 9 HVX extension: v66 10 https://developer.qualcomm.com/downloads/qualcomm-hexagon-v66-hvx-programmer-s-reference-manual 11 12We presented an overview of the project at the 2019 KVM Forum. 13 https://kvmforum2019.sched.com/event/Tmwc/qemu-hexagon-automatic-translation-of-the-isa-manual-pseudcode-to-tiny-code-instructions-of-a-vliw-architecture-niccolo-izzo-revng-taylor-simpson-qualcomm-innovation-center 14 15*** Tour of the code *** 16 17The qemu-hexagon implementation is a combination of qemu and the Hexagon 18architecture library (aka archlib). The three primary directories with 19Hexagon-specific code are 20 21 qemu/target/hexagon 22 This has all the instruction and packet semantics 23 qemu/target/hexagon/imported 24 These files are imported with very little modification from archlib 25 *.idef Instruction semantics definition 26 macros.def Mapping of macros to instruction attributes 27 encode*.def Encoding patterns for each instruction 28 iclass.def Instruction class definitions used to determine 29 legal VLIW slots for each instruction 30 qemu/target/hexagon/idef-parser 31 Parser that, given the high-level definitions of an instruction, 32 produces a C function generating equivalent tiny code instructions. 33 See README.rst. 34 qemu/linux-user/hexagon 35 Helpers for loading the ELF file and making Linux system calls, 36 signals, etc 37 38We start with scripts that generate a bunch of include files. This 39is a two step process. The first step is to use the C preprocessor to expand 40macros inside the architecture definition files. This is done in 41target/hexagon/gen_semantics.c. This step produces 42 <BUILD_DIR>/target/hexagon/semantics_generated.pyinc. 43That file is consumed by the following python scripts to produce the indicated 44header files in <BUILD_DIR>/target/hexagon 45 gen_opcodes_def.py -> opcodes_def_generated.h.inc 46 gen_op_regs.py -> op_regs_generated.h.inc 47 gen_printinsn.py -> printinsn_generated.h.inc 48 gen_op_attribs.py -> op_attribs_generated.h.inc 49 gen_helper_protos.py -> helper_protos_generated.h.inc 50 gen_shortcode.py -> shortcode_generated.h.inc 51 gen_tcg_funcs.py -> tcg_funcs_generated.c.inc 52 gen_tcg_func_table.py -> tcg_func_table_generated.c.inc 53 gen_helper_funcs.py -> helper_funcs_generated.c.inc 54 gen_idef_parser_funcs.py -> idef_parser_input.h 55 56Qemu helper functions have 3 parts 57 DEF_HELPER declaration indicates the signature of the helper 58 gen_helper_<NAME> will generate a TCG call to the helper function 59 The helper implementation 60 61Here's an example of the A2_add instruction. 62 Instruction tag A2_add 63 Assembly syntax "Rd32=add(Rs32,Rt32)" 64 Instruction semantics "{ RdV=RsV+RtV;}" 65 66By convention, the operands are identified by letter 67 RdV is the destination register 68 RsV, RtV are source registers 69 70The generator uses the operand naming conventions (see large comment in 71hex_common.py) to determine the signature of the helper function. Here are the 72results for A2_add 73 74helper_protos_generated.h.inc 75 DEF_HELPER_3(A2_add, s32, env, s32, s32) 76 77tcg_funcs_generated.c.inc 78 static void generate_A2_add( 79 CPUHexagonState *env, 80 DisasContext *ctx, 81 Insn *insn, 82 Packet *pkt) 83 { 84 TCGv RdV = tcg_temp_local_new(); 85 const int RdN = insn->regno[0]; 86 TCGv RsV = hex_gpr[insn->regno[1]]; 87 TCGv RtV = hex_gpr[insn->regno[2]]; 88 gen_helper_A2_add(RdV, cpu_env, RsV, RtV); 89 gen_log_reg_write(RdN, RdV); 90 ctx_log_reg_write(ctx, RdN); 91 tcg_temp_free(RdV); 92 } 93 94helper_funcs_generated.c.inc 95 int32_t HELPER(A2_add)(CPUHexagonState *env, int32_t RsV, int32_t RtV) 96 { 97 uint32_t slot __attribute__((unused)) = 4; 98 int32_t RdV = 0; 99 { RdV=RsV+RtV;} 100 return RdV; 101 } 102 103Note that generate_A2_add updates the disassembly context to be processed 104when the packet commits (see "Packet Semantics" below). 105 106The generator checks for fGEN_TCG_<tag> macro. This allows us to generate 107TCG code instead of a call to the helper. If defined, the macro takes 1 108argument. 109 C semantics (aka short code) 110 111This allows the code generator to override the auto-generated code. In some 112cases this is necessary for correct execution. We can also override for 113faster emulation. For example, calling a helper for add is more expensive 114than generating a TCG add operation. 115 116The gen_tcg.h file has any overrides. For example, we could write 117 #define fGEN_TCG_A2_add(GENHLPR, SHORTCODE) \ 118 tcg_gen_add_tl(RdV, RsV, RtV) 119 120The instruction semantics C code relies heavily on macros. In cases where the 121C semantics are specified only with macros, we can override the default with 122the short semantics option and #define the macros to generate TCG code. One 123example is L2_loadw_locked: 124 Instruction tag L2_loadw_locked 125 Assembly syntax "Rd32=memw_locked(Rs32)" 126 Instruction semantics "{ fEA_REG(RsV); fLOAD_LOCKED(1,4,u,EA,RdV) }" 127 128In gen_tcg.h, we use the shortcode 129#define fGEN_TCG_L2_loadw_locked(SHORTCODE) \ 130 SHORTCODE 131 132There are also cases where we brute force the TCG code generation. 133Instructions with multiple definitions are examples. These require special 134handling because qemu helpers can only return a single value. 135 136For HVX vectors, the generator behaves slightly differently. The wide vectors 137won't fit in a TCGv or TCGv_i64, so we pass TCGv_ptr variables to pass the 138address to helper functions. Here's an example for an HVX vector-add-word 139istruction. 140 static void generate_V6_vaddw( 141 CPUHexagonState *env, 142 DisasContext *ctx, 143 Insn *insn, 144 Packet *pkt) 145 { 146 const int VdN = insn->regno[0]; 147 const intptr_t VdV_off = 148 ctx_future_vreg_off(ctx, VdN, 1, true); 149 TCGv_ptr VdV = tcg_temp_local_new_ptr(); 150 tcg_gen_addi_ptr(VdV, cpu_env, VdV_off); 151 const int VuN = insn->regno[1]; 152 const intptr_t VuV_off = 153 vreg_src_off(ctx, VuN); 154 TCGv_ptr VuV = tcg_temp_local_new_ptr(); 155 const int VvN = insn->regno[2]; 156 const intptr_t VvV_off = 157 vreg_src_off(ctx, VvN); 158 TCGv_ptr VvV = tcg_temp_local_new_ptr(); 159 tcg_gen_addi_ptr(VuV, cpu_env, VuV_off); 160 tcg_gen_addi_ptr(VvV, cpu_env, VvV_off); 161 TCGv slot = tcg_constant_tl(insn->slot); 162 gen_helper_V6_vaddw(cpu_env, VdV, VuV, VvV, slot); 163 tcg_temp_free(slot); 164 gen_log_vreg_write(ctx, VdV_off, VdN, EXT_DFL, insn->slot, false); 165 ctx_log_vreg_write(ctx, VdN, EXT_DFL, false); 166 tcg_temp_free_ptr(VdV); 167 tcg_temp_free_ptr(VuV); 168 tcg_temp_free_ptr(VvV); 169 } 170 171Notice that we also generate a variable named <operand>_off for each operand of 172the instruction. This makes it easy to override the instruction semantics with 173functions from tcg-op-gvec.h. Here's the override for this instruction. 174 #define fGEN_TCG_V6_vaddw(SHORTCODE) \ 175 tcg_gen_gvec_add(MO_32, VdV_off, VuV_off, VvV_off, \ 176 sizeof(MMVector), sizeof(MMVector)) 177 178Finally, we notice that the override doesn't use the TCGv_ptr variables, so 179we don't generate them when an override is present. Here is what we generate 180when the override is present. 181 static void generate_V6_vaddw( 182 CPUHexagonState *env, 183 DisasContext *ctx, 184 Insn *insn, 185 Packet *pkt) 186 { 187 const int VdN = insn->regno[0]; 188 const intptr_t VdV_off = 189 ctx_future_vreg_off(ctx, VdN, 1, true); 190 const int VuN = insn->regno[1]; 191 const intptr_t VuV_off = 192 vreg_src_off(ctx, VuN); 193 const int VvN = insn->regno[2]; 194 const intptr_t VvV_off = 195 vreg_src_off(ctx, VvN); 196 fGEN_TCG_V6_vaddw({ fHIDE(int i;) fVFOREACH(32, i) { VdV.w[i] = VuV.w[i] + VvV.w[i] ; } }); 197 gen_log_vreg_write(ctx, VdV_off, VdN, EXT_DFL, insn->slot, false); 198 ctx_log_vreg_write(ctx, VdN, EXT_DFL, false); 199 } 200 201In addition to instruction semantics, we use a generator to create the decode 202tree. This generation is also a two step process. The first step is to run 203target/hexagon/gen_dectree_import.c to produce 204 <BUILD_DIR>/target/hexagon/iset.py 205This file is imported by target/hexagon/dectree.py to produce 206 <BUILD_DIR>/target/hexagon/dectree_generated.h.inc 207 208*** Key Files *** 209 210cpu.h 211 212This file contains the definition of the CPUHexagonState struct. It is the 213runtime information for each thread and contains stuff like the GPR and 214predicate registers. 215 216macros.h 217mmvec/macros.h 218 219The Hexagon arch lib relies heavily on macros for the instruction semantics. 220This is a great advantage for qemu because we can override them for different 221purposes. You will also notice there are sometimes two definitions of a macro. 222The QEMU_GENERATE variable determines whether we want the macro to generate TCG 223code. If QEMU_GENERATE is not defined, we want the macro to generate vanilla 224C code that will work in the helper implementation. 225 226translate.c 227 228The functions in this file generate TCG code for a translation block. Some 229important functions in this file are 230 231 gen_start_packet - initialize the data structures for packet semantics 232 gen_commit_packet - commit the register writes, stores, etc for a packet 233 decode_and_translate_packet - disassemble a packet and generate code 234 235genptr.c 236gen_tcg.h 237 238These files create a function for each instruction. It is mostly composed of 239fGEN_TCG_<tag> definitions followed by including tcg_funcs_generated.c.inc. 240 241op_helper.c 242 243This file contains the implementations of all the helpers. There are a few 244general purpose helpers, but most of them are generated by including 245helper_funcs_generated.c.inc. There are also several helpers used for debugging. 246 247 248*** Packet Semantics *** 249 250VLIW packet semantics differ from serial semantics in that all input operands 251are read, then the operations are performed, then all the results are written. 252For exmaple, this packet performs a swap of registers r0 and r1 253 { r0 = r1; r1 = r0 } 254Note that the result is different if the instructions are executed serially. 255 256Packet semantics dictate that we defer any changes of state until the entire 257packet is committed. We record the results of each instruction in a side data 258structure, and update the visible processor state when we commit the packet. 259 260The data structures are divided between the runtime state and the translation 261context. 262 263During the TCG generation (see translate.[ch]), we use the DisasContext to 264track what needs to be done during packet commit. Here are the relevant 265fields 266 267 reg_log list of registers written 268 reg_log_idx index into ctx_reg_log 269 pred_log list of predicates written 270 pred_log_idx index into ctx_pred_log 271 store_width width of stores (indexed by slot) 272 273During runtime, the following fields in CPUHexagonState (see cpu.h) are used 274 275 new_value new value of a given register 276 reg_written boolean indicating if register was written 277 new_pred_value new value of a predicate register 278 pred_written boolean indicating if predicate was written 279 mem_log_stores record of the stores (indexed by slot) 280 281For Hexagon Vector eXtensions (HVX), the following fields are used 282 VRegs Vector registers 283 future_VRegs Registers to be stored during packet commit 284 tmp_VRegs Temporary registers *not* stored during commit 285 VRegs_updated Mask of predicated vector writes 286 QRegs Q (vector predicate) registers 287 future_QRegs Registers to be stored during packet commit 288 QRegs_updated Mask of predicated vector writes 289 290*** Debugging *** 291 292You can turn on a lot of debugging by changing the HEX_DEBUG macro to 1 in 293internal.h. This will stream a lot of information as it generates TCG and 294executes the code. 295 296To track down nasty issues with Hexagon->TCG generation, we compare the 297execution results with actual hardware running on a Hexagon Linux target. 298Run qemu with the "-d cpu" option. Then, we can diff the results and figure 299out where qemu and hardware behave differently. 300 301The stacks are located at different locations. We handle this by changing 302env->stack_adjust in translate.c. First, set this to zero and run qemu. 303Then, change env->stack_adjust to the difference between the two stack 304locations. Then rebuild qemu and run again. That will produce a very 305clean diff. 306 307Here are some handy places to set breakpoints 308 309 At the call to gen_start_packet for a given PC (note that the line number 310 might change in the future) 311 br translate.c:602 if ctx->base.pc_next == 0xdeadbeef 312 The helper function for each instruction is named helper_<TAG>, so here's 313 an example that will set a breakpoint at the start 314 br helper_A2_add 315 If you have the HEX_DEBUG macro set, the following will be useful 316 At the start of execution of a packet for a given PC 317 br helper_debug_start_packet if env->gpr[41] == 0xdeadbeef 318 At the end of execution of a packet for a given PC 319 br helper_debug_commit_end if env->this_PC == 0xdeadbeef 320