1 The Linux Test Project (LTP) provides the basis for testing the lx API 2 implementation. The project webpage is at http://linux-test-project.github.io/ 3 and the source is available on GitHub at 4 https://github.com/linux-test-project/ltp.git. 5 6 LTP should be built and run from within an lx zone. 7 8 To build 9 -------- 10 As root first make sure you have the tools installed: 11 apt-get install build-essential autoconf automake git quota 12 13 Additional prerequisites are required activate some tests: 14 apt-get install attr-dev libaio-dev 15 16 As a normal user: 17 git clone https://github.com/linux-test-project/ltp.git 18 cd ltp 19 make autotools 20 ./configure 21 make all 22 23 As root: 24 make install 25 26 The tests can be built in both a zone that has been installed with a 32-bit 27 version of Linux and another zone that has been installed with a 64-bit version 28 of the same release. When testing the 64-bit zone, a copy of the 32-bit build 29 can be run in the 64-bit zone to ensure that 32-bit applications work 30 correctly on a 64-bit install. 31 32 Running the tests 33 ----------------- 34 The LTP source tree provides detailed documentation on using the test suite, so 35 this readme only give a short summary. 36 37 Because many of the tests are targetted at kernel functionality which does not 38 apply to Illumos, or test capabilities which are not available from within a 39 zone, or test system call functionality which has not yet been completed, a 40 skip list is used during the test run to bypass tests which are known to fail. 41 42 The skip list lives in this directory and is delivered on the system. It is 43 available from within the lx zone as /native/usr/lib/brand/lx/ltp_skiplist. As 44 new functionality is completed, the skip list should be updated to remove tests 45 which now work. 46 47 As root: 48 cd /opt/ltp 49 /opt/ltp/runltp -f `cat /native/usr/lib/brand/lx/ltp_tests` \ 50 -S /native/usr/lib/brand/lx/ltp_skiplist -p >/tmp/test.log 51 52 When the test run has finished, the results will be logged in a date/time 53 stamped file under /opt/ltp/results. The summary at the end of the log file 54 should show "Total Failures: 0". If not, something is wrong. 55 56 Running tests for development 57 ----------------------------- 58 The source for the tests can be found under the testcases directory. The 59 largest and most useful set for lx live under testcases/kernel/syscalls. 60 61 For development purposes, an individual test (or tests) can be run by listing 62 them in a command file, one per line. For example, with a command file named 63 ~/tcmds, to run the read01 test you setup the file so it looks like this: 64 read01 read01 65 66 You can run that specific test as follows: 67 /opt/ltp/runltp -f ~/tcmds -p -l ~/read.log 68 69 Test status 70 ----------- 71 This section provides a short summary of the rationale for why tests are being 72 skipped. 73 74 LTP groups tests into command files (i.e. syscalls, nptl, etc. provided with 75 the -f option in the runltp command shown above). A complete list of the groups 76 can be seen in LTP source tree under the runtest directory. Some of these 77 groups are obviously not applicable when running in an lx zone. The remaining 78 groups still need work before they can be run. The groups shown in the runltp 79 command above are expected to work when the skip list is used. The 'syscalls' 80 command file runs the majority of the actual system call tests which we are 81 interested in. 82 83 The following table indicates why specific subtests are being skipped. Also 84 note that the following tests pass in a 64-bit lx zone, but fail in a zone 85 installed with a 32-bit Linux build: mmap15, open12, openat02 and sendfile09. 86 87 Legend: 88 x = never in a zone 89 * = fails on kvm and bare metal too 90 # = emulation not implemented yet 91 - = could enable with a test zone config change 92 93 - access06 wants a block device 94 x acct01 enables bsd process accounting 95 # add_key01 96 # add_key02 97 x adjtimex01 98 x adjtimex02 99 x bdflush01 100 x cacheflush01 101 x chmod03 need PRIV_SYS_CONFIG to set sticky bit on reg file 102 - chmod06 needs dev 103 x chmod07 need PRIV_SYS_CONFIG to set sticky bit on reg file 104 - chown04 needs dev 105 - chown04_16 needs dev 106 # clone02 107 # clone08 108 - creat06 wants to mount a ro fs 109 x creat07 we don't behave this way for ETXTBSY 110 x creat08 sets euid to 'nobody', loses PRIV_FILE_SETID to set sgid 111 x execve04 we don't behave this way for ETXTBSY 112 # fallocate01 113 # fallocate02 114 # fallocate03 115 x fchmod02 need PRIV_SYS_CONFIG to set sticky bit on reg file 116 x fchmod03 need PRIV_SYS_CONFIG to set sticky bit on reg file 117 - fchmod06 needs dev 118 # fchown04 mounts 119 # fchown04_16 120 # fcntl06 not supported on linux 121 # fcntl06_64 122 # fcntl23 leases not implemented 123 # fcntl23_64 " 124 # fcntl24 " 125 # fcntl24_64 " 126 # fcntl25 " 127 # fcntl25_64 " 128 # fcntl26 " 129 # fcntl26_64 " 130 # fcntl30 131 # fcntl30_64 132 # fcntl31 setown/getown not impl 133 # fcntl31_64 134 # fcntl32 F_SETLEASE not impl 135 # fcntl32_64 136 # fcntl33 F_SETLEASE not impl 137 # fcntl33_64 138 # fork05 asm into %fs reg 139 - fork09 needs a swap cap of ~9GB 140 # fork13 decided not to support this 141 # fork14 " 142 # ftruncate04 need a mnt with mandatory locking 143 # ftruncate04_64 144 # getdents02 wrong errno on test 4 - perf. impact too high 145 # getdents02_64 146 # get_mempolicy01 147 x getrusage03 we don't fill in the ru_maxrss field 148 - getxattr01 need attr/xattr.h at build time 149 - getxattr02 150 - getxattr03 151 # ioctl03 needs /dev/net/tun 152 # io_cancel01 libaio stuff not done 153 # io_destroy01 154 # io_getevents01 155 # io_setup01 156 # io_submit01 157 - inotify03 needs dev 158 # fanotify01 don't have fanotify 159 # fanotify02 160 # fanotify03 161 # fanotify04 162 # fanotify05 163 # keyctl01 no kernel keyring support 164 - lchown03 needs to mount ro fs 165 - lchown03_16 166 - linkat02 needs dev 167 - link08 needs dev 168 x mem01 crashme test which expects OOM killer to save the day 169 - mkdir03 needs dev 170 - mkdirat02 needs dev 171 x mknod01 makes block and chr devs 172 - mknod07 needs dev 173 - mknodat02 needs dev 174 # mmap13 expects "invalid access" to SIGBUS 175 - mount01 needs dev 176 - mount02 needs dev 177 x mount03 mounts ext2 178 - mount04 needs dev 179 x mount05 mounts ext2 180 x mount06 mounts ext2 181 # mq_notify01 182 # mq_notify02 183 # mq_open01 184 # mq_timedreceive01 185 # mq_timedsend01 186 # mq_unlink01 187 x mremap01 188 x mremap02 189 x mremap03 190 x mremap04 191 x mremap05 192 # msgctl12 uses MSG_STAT 193 # msgrcv07 MSG_EXCEPT subtest - not avail. 194 x open01 need PRIV_SYS_CONFIG to set sticky bit on reg file 195 # open02 expects NOATIME to cause err for unpriv user 196 # open10 setgid on sgid subdir behavior 197 x open11 makes device 198 # ppoll01 199 # process_vm_readv01 200 # process_vm_readv02 201 # process_vm_readv03 202 # process_vm_writev01 203 # process_vm_writev02 204 # prot_hsymlinks /proc/sys/fs/protected_hardlinks 205 x ptrace04 not supp on our arch 206 # ptrace05 OS-3307 207 # read02 checks errno for O_DIRECT 208 # readahead01 209 # readahead02 210 # readdir21 dir perf. issue 211 - rename11 needs dev 212 - renameat01 needs dev 213 - rmdir02 needs dev 214 x sched_getparam01 assumes Linux SCHED_OTHER return value 215 x sched_getparam02 assumes Linux SCHED_OTHER return value 216 # sched_rr_get_interval01 217 # sched_rr_get_interval02 218 # sched_rr_get_interval03 219 x sched_setparam02 tries to set Linux policies 220 x sched_setparam03 tries to set Linux policies 221 # sched_getscheduler01 222 # sched_getscheduler02 223 x sched_setscheduler01 224 x sched_setaffinity01 225 x sched_getaffinity01 226 # semctl01 all pass but SEM_STAT - linux specific 227 # semop02 last test fails errno - expensive 228 # sendfile02 OS-3296 229 # sendfile02_64 " 230 # sendfile04 " 231 # sendfile04_64 " 232 # sendfile06 " 233 # sendfile06_64 " 234 # sendfile07 " 235 # sendfile07_64 " 236 sendmsg01 OS-3295 - tests actually pass 237 x setfsuid04 no real equiv. and only for NFS server 238 x setfsuid04_16 239 # sgetmask01 obsolete 240 # setgroups04_16 expects sig11 for certain err 241 # setns01 242 # setns02 243 # setpgid02 all pass but one, expects pid 0 to be there 244 * setregid02 fails on bare metal, expects to lookup group "nobody" which 245 * setregid02_16 doesn't exist. it is "nogroup" on ubuntu at least 246 # setrlimit01 all pass but one, expects to set proc limit 247 x settimeofday01 248 # setxattr01 249 # setxattr02 250 # setxattr03 251 # shmget05 OS-3326 252 # splice01 253 # splice02 254 # splice03 255 # tee01 256 # tee02 257 # ssetmask01 obsolete 258 x stime01 259 x switch01 260 # sync_file_range01 261 # sysconf01 most pass but see OS-3305 262 # sysctl01 the build compiled this out, 263 # sysctl03 this syscall is basically obsolete 264 # sysctl04 obsolete 265 # sysctl05 266 # syslog01 267 # syslog02 268 # syslog03 269 # syslog04 270 # syslog05 271 # syslog06 272 # syslog07 273 # syslog08 274 # syslog09 275 # syslog10 276 # syslog11 277 # syslog12 278 # unshare01 279 # unshare02 280 - umount01 needs dev 281 - umount02 needs dev 282 - umount03 needs dev 283 x ustat01 obsolete call to stat FS 284 x ustat02 obsolete call to stat FS 285 - utime06 needs dev 286 - utimes01 needs dev 287 # utimensat01 many subtests pass but see OS-3328 for the rest 288 # vmsplice01 289 # vmsplice02 290 # perf_event_open01 291 # perf_event_open02