Fluent Essays

Machine learning with Python, need code , and write report to analysis algorithms age anaemia creatinine_phosphokinase diabetes ejection_fraction high_blood_pressure platelets serum_creatinine serum_sodium sex smoking time DEATH_EVENT 75 0 582 0 20 1 265000 1.9 130 1 0 4 1 55 0 7861 0 38 0 263358.03 1.1 136 1 0 6 1 65 0 146 0 20 0 162000 1.3 129 1 1 7 1 50 1 111 0 20 0 210000 1.9 137 1 0 7 1 65 1 160 1 20 0 327000 2.7 116 0 0 8 1 90 1 47 0 40 1 204000 2.1 132 1 1 8 1 75 1 246 0 15 0 127000 1.2 137 1 0 10 1 60 1 315 1 60 0 454000 1.1 131 1 1 10 1 65 0 157 0 65 0 263358.03 1.5 138 0 0 10 1 80 1 123 0 35 1 388000 9.4 133 1 1 10 1 75 1 81 0 38 1 368000 4 131 1 1 10 1 62 0 231 0 25 1 253000 0.9 140 1 1 10 1 45 1 981 0 30 0 136000 1.1 137 1 0 11 1 50 1 168 0 38 1 276000 1.1 137 1 0 11 1 49 1 80 0 30 1 427000 1 138 0 0 12 0 82 1 379 0 50 0 47000 1.3 136 1 0 13 1 87 1 149 0 38 0 262000 0.9 140 1 0 14 1 45 0 582 0 14 0 166000 0.8 127 1 0 14 1 70 1 125 0 25 1 237000 1 140 0 0 15 1 48 1 582 1 55 0 87000 1.9 121 0 0 15 1 65 1 52 0 25 1 276000 1.3 137 0 0 16 0 65 1 128 1 30 1 297000 1.6 136 0 0 20 1 68 1 220 0 35 1 289000 0.9 140 1 1 20 1 53 0 63 1 60 0 368000 0.8 135 1 0 22 0 75 0 582 1 30 1 263358.03 1.83 134 0 0 23 1 80 0 148 1 38 0 149000 1.9 144 1 1 23 1 95 1 112 0 40 1 196000 1 138 0 0 24 1 70 0 122 1 45 1 284000 1.3 136 1 1 26 1 58 1 60 0 38 0 153000 5.8 134 1 0 26 1 82 0 70 1 30 0 200000 1.2 132 1 1 26 1 94 0 582 1 38 1 263358.03 1.83 134 1 0 27 1 85 0 23 0 45 0 360000 3 132 1 0 28 1 50 1 249 1 35 1 319000 1 128 0 0 28 1 50 1 159 1 30 0 302000 1.2 138 0 0 29 0 65 0 94 1 50 1 188000 1 140 1 0 29 1 69 0 582 1 35 0 228000 3.5 134 1 0 30 1 90 1 60 1 50 0 226000 1 134 1 0 30 1 82 1 855 1 50 1 321000 1 145 0 0 30 1 60 0 2656 1 30 0 305000 2.3 137 1 0 30 0 60 0 235 1 38 0 329000 3 142 0 0 30 1 70 0 582 0 20 1 263358.03 1.83 134 1 1 31 1 50 0 124 1 30 1 153000 1.2 136 0 1 32 1 70 0 571 1 45 1 185000 1.2 139 1 1 33 1 72 0 127 1 50 1 218000 1 134 1 0 33 0 60 1 588 1 60 0 194000 1.1 142 0 0 33 1 50 0 582 1 38 0 310000 1.9 135 1 1 35 1 51 0 1380 0 25 1 271000 0.9 130 1 0 38 1 60 0 582 1 38 1 451000 0.6 138 1 1 40 1 80 1 553 0 20 1 140000 4.4 133 1 0 41 1 57 1 129 0 30 0 395000 1 140 0 0 42 1 68 1 577 0 25 1 166000 1 138 1 0 43 1 53 1 91 0 20 1 418000 1.4 139 0 0 43 1 60 0 3964 1 62 0 263358.03 6.8 146 0 0 43 1 70 1 69 1 50 1 351000 1 134 0 0 44 1 60 1 260 1 38 0 255000 2.2 132 0 1 45 1 95 1 371 0 30 0 461000 2 132 1 0 50 1 70 1 75 0 35 0 223000 2.7 138 1 1 54 0 60 1 607 0 40 0 216000 0.6 138 1 1 54 0 49 0 789 0 20 1 319000 1.1 136 1 1 55 1 72 0 364 1 20 1 254000 1.3 136 1 1 59 1 45 0 7702 1 25 1 390000 1 139 1 0 60 1 50 0 318 0 40 1 216000 2.3 131 0 0 60 1 55 0 109 0 35 0 254000 1.1 139 1 1 60 0 45 0 582 0 35 0 385000 1 145 1 0 61 1 45 0 582 0 80 0 263358.03 1.18 137 0 0 63 0 60 0 68 0 20 0 119000 2.9 127 1 1 64 1 42 1 250 1 15 0 213000 1.3 136 0 0 65 1 72 1 110 0 25 0 274000 1 140 1 1 65 1 70 0 161 0 25 0 244000 1.2 142 0 0 66 1 65 0 113 1 25 0 497000 1.83 135 1 0 67 1 41 0 148 0 40 0 374000 0.8 140 1 1 68 0 58 0 582 1 35 0 122000 0.9 139 1 1 71 0 85 0 5882 0 35 0 243000 1 132 1 1 72 1 65 0 224 1 50 0 149000 1.3 137 1 1 72 0 69 0 582 0 20 0 266000 1.2 134 1 1 73 1 60 1 47 0 20 0 204000 0.7 139 1 1 73 1 70 0 92 0 60 1 317000 0.8 140 0 1 74 0 42 0 102 1 40 0 237000 1.2 140 1 0 74 0 75 1 203 1 38 1 283000 0.6 131 1 1 74 0 55 0 336 0 45 1 324000 0.9 140 0 0 74 0 70 0 69 0 40 0 293000 1.7 136 0 0 75 0 67 0 582 0 50 0 263358.03 1.18 137 1 1 76 0 60 1 76 1 25 0 196000 2.5 132 0 0 77 1 79 1 55 0 50 1 172000 1.8 133 1 0 78 0 59 1 280 1 25 1 302000 1 141 0 0 78 1 51 0 78 0 50 0 406000 0.7 140 1 0 79 0 55 0 47 0 35 1 173000 1.1 137 1 0 79 0 65 1 68 1 60 1 304000 0.8 140 1 0 79 0 44 0 84 1 40 1 235000 0.7 139 1 0 79 0 57 1 115 0 25 1 181000 1.1 144 1 0 79 0 70 0 66 1 45 0 249000 0.8 136 1 1 80 0 60 0 897 1 45 0 297000 1 133 1 0 80 0 42 0 582 0 60 0 263358.03 1.18 137 0 0 82 0 60 1 154 0 25 0 210000 1.7 135 1 0 82 1 58 0 144 1 38 1 327000 0.7 142 0 0 83 0 58 1 133 0 60 1 219000 1 141 1 0 83 0 63 1 514 1 25 1 254000 1.3 134 1 0 83 0 70 1 59 0 60 0 255000 1.1 136 0 0 85 0 60 1 156 1 25 1 318000 1.2 137 0 0 85 0 63 1 61 1 40 0 221000 1.1 140 0 0 86 0 65 1 305 0 25 0 298000 1.1 141 1 0 87 0 75 0 582 0 45 1 263358.03 1.18 137 1 0 87 0 80 0 898 0 25 0 149000 1.1 144 1 1 87 0 42 0 5209 0 30 0 226000 1 140 1 1 87 0 60 0 53 0 50 1 286000 2.3 143 0 0 87 0 72 1 328 0 30 1 621000 1.7 138 0 1 88 1 55 0 748 0 45 0 263000 1.3 137 1 0 88 0 45 1 1876 1 35 0 226000 0.9 138 1 0 88 0 63 0 936 0 38 0 304000 1.1 133 1 1 88 0 45 0 292 1 35 0 850000 1.3 142 1 1 88 0 85 0 129 0 60 0 306000 1.2 132 1 1 90 1 55 0 60 0 35 0 228000 1.2 135 1 1 90 0 50 0 369 1 25 0 252000 1.6 136 1 0 90 0 70 1 143 0 60 0 351000 1.3 137 0 0 90 1 60 1 754 1 40 1 328000 1.2 126 1 0 91 0 58 1 400 0 40 0 164000 1 139 0 0 91 0 60 1 96 1 60 1 271000 0.7 136 0 0 94 0 85 1 102 0 60 0 507000 3.2 138 0 0 94 0 65 1 113 1 60 1 203000 0.9 140 0 0 94 0 86 0 582 0 38 0 263358.03 1.83 134 0 0 95 1 60 1 737 0 60 1 210000 1.5 135 1 1 95 0 66 1 68 1 38 1 162000 1 136 0 0 95 0 60 0 96 1 38 0 228000 0.75 140 0 0 95 0 60 1 582 0 30 1 127000 0.9 145 0 0 95 0 60 0 582 0 40 0 217000 3.7 134 1 0 96 1 43 1 358 0 50 0 237000 1.3 135 0 0 97 0 46 0 168 1 17 1 271000 2.1 124 0 0 100 1 58 1 200 1 60 0 300000 0.8 137 0 0 104 0 61 0 248 0 30 1 267000 0.7 136 1 1 104 0 53 1 270 1 35 0 227000 3.4 145 1 0 105 0 53 1 1808 0 60 1 249000 0.7 138 1 1 106 0 60 1 1082 1 45 0 250000 6.1 131 1 0 107 0 46 0 719 0 40 1 263358.03 1.18 137 0 0 107 0 63 0 193 0 60 1 295000 1.3 145 1 1 107 0 81 0 4540 0 35 0 231000 1.18 137 1 1 107 0 75 0 582 0 40 0 263358.03 1.18 137 1 0 107 0 65 1 59 1 60 0 172000 0.9 137 0 0 107 0 68 1 646 0 25 0 305000 2.1 130 1 0 108 0 62 0 281 1 35 0 221000 1 136 0 0 108 0 50 0 1548 0 30 1 211000 0.8 138 1 0 108 0 80 0 805 0 38 0 263358.03 1.1 134 1 0 109 1 46 1 291 0 35 0 348000 0.9 140 0 0 109 0 50 0 482 1 30 0 329000 0.9 132 0 0 109 0 61 1 84 0 40 1 229000 0.9 141 0 0 110 0 72 1 943 0 25 1 338000 1.7 139 1 1 111 1 50 0 185 0 30 0 266000 0.7 141 1 1 112 0 52 0 132 0 30 0 218000 0.7 136 1 1 112 0 64 0 1610 0 60 0 242000 1 137 1 0 113 0 75 1 582 0 30 0 225000 1.83 134 1 0 113 1 60 0 2261 0 35 1 228000 0.9 136 1 0 115 0 72 0 233 0 45 1 235000 2.5 135 0 0 115 1 62 0 30 1 60 1 244000 0.9 139 1 0 117 0 50 0 115 0 45 1 184000 0.9 134 1 1 118 0 50 0 1846 1 35 0 263358.03 1.18 137 1 1 119 0 65 1 335 0 35 1 235000 0.8 136 0 0 120 0 60 1 231 1 25 0 194000 1.7 140 1 0 120 0 52 1 58 0 35 0 277000 1.4 136 0 0 120 0 50 0 250 0 25 0 262000 1 136 1 1 120 0 85 1 910 0 50 0 235000 1.3 134 1 0 121 0 59 1 129 0 45 1 362000 1.1 139 1 1 121 0 66 1 72 0 40 1 242000 1.2 134 1 0 121 0 45 1 130 0 35 0 174000 0.8 139 1 1 121 0 63 1 582 0 40 0 448000 0.9 137 1 1 123 0 50 1 2334 1 35 0 75000 0.9 142 0 0 126 1 45 0 2442 1 30 0 334000 1.1 139 1 0 129 1 80 0 776 1 38 1 192000 1.3 135 0 0 130 1 53 0 196 0 60 0 220000 0.7 133 1 1 134 0 59 0 66 1 20 0 70000 2.4 134 1 0 135 1 65 0 582 1 40 0 270000 1 138 0 0 140 0 70 0 835 0 35 1 305000 0.8 133 0 0 145 0 51 1 582 1 35 0 263358.03 1.5 136 1 1 145 0 52 0 3966 0 40 0 325000 0.9 140 1 1 146 0 70 1 171 0 60 1 176000 1.1 145 1 1 146 0 50 1 115 0 20 0 189000 0.8 139 1 0 146 0 65 0 198 1 35 1 281000 0.9 137 1 1 146 0 60 1 95 0 60 0 337000 1 138 1 1 146 0 69 0 1419 0 40 0 105000 1 135 1 1 147 0 49 1 69 0 50 0 132000 1 140 0 0 147 0 63 1 122 1 60 0 267000 1.2 145 1 0 147 0 55 0 835 0 40 0 279000 0.7 140 1 1 147 0 40 0 478 1 30 0 303000 0.9 136 1 0 148 0 59 1 176 1 25 0 221000 1 136 1 1 150 1 65 0 395 1 25 0 265000 1.2 136 1 1 154 1 75 0 99 0 38 1 224000 2.5 134 1 0 162 1 58 1 145 0 25 0 219000 1.2 137 1 1 170 1 60.667 1 104 1 30 0 389000 1.5 136 1 0 171 1 50 0 582 0 50 0 153000 0.6 134 0 0 172 1 60 0 1896 1 25 0 365000 2.1 144 0 0 172 1 60.667 1 151 1 40 1 201000 1 136 0 0 172 0 40 0 244 0 45 1 275000 0.9 140 0 0 174 0 80 0 582 1 35 0 350000 2.1 134 1 0 174 0 64 1 62 0 60 0 309000 1.5 135 0 0 174 0 50 1 121 1 40 0 260000 0.7 130 1 0 175 0 73 1 231 1 30 0 160000 1.18 142 1 1 180 0 45 0 582 0 20 1 126000 1.6 135 1 0 180 1 77 1 418 0 45 0 223000 1.8 145 1 0 180 1 45 0 582 1 38 1 263358.03 1.18 137 0 0 185 0 65 0 167 0 30 0 259000 0.8 138 0 0 186 0 50 1 582 1 20 1 279000 1 134 0 0 186 0 60 0 1211 1 35 0 263358.03 1.8 113 1 1 186 0 63 1 1767 0 45 0 73000 0.7 137 1 0 186 0 45 0 308 1 60 1 377000 1 136 1 0 186 0 70 0 97 0 60 1 220000 0.9 138 1 0 186 0 60 0 59 0 25 1 212000 3.5 136 1 1 187 0 78 1 64 0 40 0 277000 0.7 137 1 1 187 0 50 1 167 1 45 0 362000 1 136 0 0 187 0 40 1 101 0 40 0 226000 0.8 141 0 0 187 0 85 0 212 0 38 0 186000 0.9 136 1 0 187 0 60 1 2281 1 40 0 283000 1 141 0 0 187 0 49 0 972 1 35 1 268000 0.8 130 0 0 187 0 70 0 212 1 17 1 389000 1 136 1 1 188 0 50 0 582 0 62 1 147000 0.8 140 1 1 192 0 78 0 224 0 50 0 481000 1.4 138 1 1 192 0 48 1 131 1 30 1 244000 1.6 130 0 0 193 1 65 1 135 0 35 1 290000 0.8 134 1 0 194 0 73 0 582 0 35 1 203000 1.3 134 1 0 195 0 70 0 1202 0 50 1 358000 0.9 141 0 0 196 0 54 1 427 0 70 1 151000 9 137 0 0 196 1 68 1 1021 1 35 0 271000 1.1 134 1 0 197 0 55 0 582 1 35 1 371000 0.7 140 0 0 197 0 73 0 582 0 20 0 263358.03 1.83 134 1 0 198 1 65 0 118 0 50 0 194000 1.1 145 1 1 200 0 42 1 86 0 35 0 365000 1.1 139 1 1 201 0 47 0 582 0 25 0 130000 0.8 134 1 0 201 0 58 0 582 1 25 0 504000 1 138 1 0 205 0 75 0 675 1 60 0 265000 1.4 125 0 0 205 0 58 1 57 0 25 0 189000 1.3 132 1 1 205 0 55 1 2794 0 35 1 141000 1 140 1 0 206 0 65 0 56 0 25 0 237000 5 130 0 0 207 0 72 0 211 0 25 0 274000 1.2 134 0 0 207 0 60 0 166 0 30 0 62000 1.7 127 0 0 207 1 70 0 93 0 35 0 185000 1.1 134 1 1 208 0 40 1 129 0 35 0 255000 0.9 137 1 0 209 0 53 1 707 0 38 0 330000 1.4 137 1 1 209 0 53 1 582 0 45 0 305000 1.1 137 1 1 209 0 77 1 109 0 50 1 406000 1.1 137 1 0 209 0 75 0 119 0 50 1 248000 1.1 148 1 0 209 0 70 0 232 0 30 0 173000 1.2 132 1 0 210 0 65 1 720 1 40 0 257000 1 136 0 0 210 0 55 1 180 0 45 0 263358.03 1.18 137 1 1 211 0 70 0 81 1 35 1 533000 1.3 139 0 0 212 0 65 0 582 1 30 0 249000 1.3 136 1 1 212 0 40 0 90 0 35 0 255000 1.1 136 1 1 212 0 73 1 1185 0 40 1 220000 0.9 141 0 0 213 0 54 0 582 1 38 0 264000 1.8 134 1 0 213 0 61 1 80 1 38 0 282000 1.4 137 1 0 213 0 55 0 2017 0 25 0 314000 1.1 138 1 0 214 1 64 0 143 0 25 0 246000 2.4 135 1 0 214 0 40 0 624 0 35 0 301000 1 142 1 1 214 0 53 0 207 1 40 0 223000 1.2 130 0 0 214 0 50 0 2522 0 30 1 404000 0.5 139 0 0 214 0 55 0 572 1 35 0 231000 0.8 143 0 0 215 0 50 0 245 0 45 1 274000 1 133 1 0 215 0 70 0 88 1 35 1 236000 1.2 132 0 0 215 0 53 1 446 0 60 1 263358.03 1 139 1 0 215 0 52 1 191 1 30 1 334000 1 142 1 1 216 0 65 0 326 0 38 0 294000 1.7 139 0 0 220 0 58 0 132 1 38 1 253000 1 139 1 0 230 0 45 1 66 1 25 0 233000 0.8 135 1 0 230 0 53 0 56 0 50 0 308000 0.7 135 1 1 231 0 55 0 66 0 40 0 203000 1 138 1 0 233 0 62 1 655 0 40 0 283000 0.7 133 0 0 233 0 65 1 258 1 25 0 198000 1.4 129 1 0 235 1 68 1 157 1 60 0 208000 1 140 0 0 237 0 61 0 582 1 38 0 147000 1.2 141 1 0 237 0 50 1 298 0 35 0 362000 0.9 140 1 1 240 0 55 0 1199 0 20 0 263358.03 1.83 134 1 1 241 1 56 1 135 1 38 0 133000 1.7 140 1 0 244 0 45 0 582 1 38 0 302000 0.9 140 0 0 244 0 40 0 582 1 35 0 222000 1 132 1 0 244 0 44 0 582 1 30 1 263358.03 1.6 130 1 1 244 0 51 0 582 1 40 0 221000 0.9 134 0 0 244 0 67 0 213 0 38 0 215000 1.2 133 0 0 245 0 42 0 64 0 40 0 189000 0.7 140 1 0 245 0 60 1 257 1 30 0 150000 1 137 1 1 245 0 45 0 582 0 38 1 422000 0.8 137 0 0 245 0 70 0 618 0 35 0 327000 1.1 142 0 0 245 0 70 0 582 1 38 0 25100 1.1 140 1 0 246 0 50 1 1051 1 30 0 232000 0.7 136 0 0 246 0 55 0 84 1 38 0 451000 1.3 136 0 0 246 0 70 0 2695 1 40 0 241000 1 137 1 0 247 0 70 0 582 0 40 0 51000 2.7 136 1 1 250 0 42 0 64 0 30 0 215000 3.8 128 1 1 250 0 65 0 1688 0 38 0 263358.03 1.1 138 1 1 250 0 50 1 54 0 40 0 279000 0.8 141 1 0 250 0 55 1 170 1 40 0 336000 1.2 135 1 0 250 0 60 0 253 0 35 0 279000 1.7 140 1 0 250 0 45 0 582 1 55 0 543000 1 132 0 0 250 0 65 0 892 1 35 0 263358.03 1.1 142 0 0 256 0 90 1 337 0 38 0 390000 0.9 144 0 0 256 0 45 0 615 1 55 0 222000 0.8 141 0 0 257 0 60 0 320 0 35 0 133000 1.4 139 1 0 258 0 52 0 190 1 38 0 382000 1 140 1 1 258 0 63 1 103 1 35 0 179000 0.9 136 1 1 270 0 62 0 61 1 38 1 155000 1.1 143 1 1 270 0 55 0 1820 0 38 0 270000 1.2 139 0 0 271 0 45 0 2060 1 60 0 742000 0.8 138 0 0 278 0 45 0 2413 0 38 0 140000 1.4 140 1 1 280 0 50 0 196 0 45 0 395000 1.6 136 1 1 285 0 Background In this project you are given a dataset and an article that uses this dataset. The authors have developed ten ML models for predicting survival of patients with heart failure and compared their performance. You must read the article to understand the problem, the dataset, and the methodology to complete the following tasks. Dataset The dataset contains the medical records of patients who had heart failure, collected during their follow-up period. Each patient profile has 13 clinical features. A detailed description of the dataset can be found in the Dataset section of the provided article (patient_survival_prediction.pdf). Tasks: 1. Read the article and reproduce the results presented in Table-4 using Python modules and packages (including your own script or customised codes). Write a report summarising the dataset, used ML methods, experiment protocol and results including variations, if any. During reproducing the results: i) you should use the same set of features used by the authors. ii) you should use the same classifier with exact parameter values. iii) you should use the same training/test splitting approach as used by the authors. iv) you should use the same pre/post processing, if any, used by the authors. v) you should report the same performance metrics as shown in Table-4. N.B. (i) Some of the ML methods are not covered in the current unit. Consider them as HD tasks i.e., based on the knowledge gained in the unit you should be able to find necessary packages and modules to reproduce the results. (ii) If you find any issue in reproducing results or some subtle variations are found due to implementation differences of packages and modules in Python then appropriate explanation of them will be considered during evaluation of your submission. (iii) Similarly, variation in results due to randomness of data splitting will also be considered during evaluation based on your explanation. (iii) Obtained marks will be proportional to the number of ML methods that you will report in your submission with correctly reproduced results. (iv) Make sure your Python code segment generates the reported results, otherwise you will receive zero marks for this task. Criteria: appropriately implemented >=90% of the methods presented in the article. Variation of marks in this group will depend on the quality of report. 2. Design and develop your own ML solution for this problem. The proposed solution should be different from all approaches mentioned in the provided article. This does not mean that you must have to choose a new ML algorithm. You can develop a novel solution by changing the feature selection approach or parameter optimisations process of used ML methods or using different ML methods or different combinations of them. This means, the proposed system should be substantially different from the methods presented in the article but not limited to only change of ML methods. Compare the result with reported methods in the article. Write a technical report summarising your solution design and outcomes. The report should include: i) Motivation behind the proposed solution. ii) How the proposed solution is different from existing ones. iii) Detail description of the model including all parameters so that any reader can implement your model. iv) Description of experimental protocol. v) Evaluation metrics. vi) Present results using tables and graphs. vii) Compare and discuss results with respect to existing literatures. viii) Appropriate references (IEEE numbered). Criteria: an appropriate solution presented whose performance is better than the best reported performances in the article (Table 11). The variation in the marking in this group will depend on the quality of the report. 3. Present your result in a 3 minutes video using PowerPoint slides/animation. Chicco and Jurman BMC Medical Informatics and Decision Making (2020) 20:16 https://doi.org/10.1186/s12911-020-1023-5 RESEARCH ARTICLE Open Access Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone Davide Chicco1* and Giuseppe Jurman2 Abstract Background: Cardiovascular diseases kill approximately 17 million people globally every year, and they mainly exhibit as myocardial infarctions and heart failures. Heart failure (HF) occurs when the heart cannot pump enough blood to meet the needs of the body. Available electronic medical records of patients quantify symptoms, body features, and clinical laboratory test values, which can be used to perform biostatistics analysis aimed at highlighting patterns and correlations otherwise undetectable by medical doctors. Machine learning, in particular, can predict patients’ survival from their data and can individuate the most important features among those included in their medical records. Methods: In this paper, we analyze a dataset of 299 patients with heart failure collected in 2015. We apply several machine learning classifiers to both predict the patients survival, and rank the features corresponding to the most important risk factors. We also perform an alternative feature ranking analysis by employing traditional biostatistics tests, and compare these results with those provided by the machine learning algorithms. Since both feature ranking approaches clearly identify serum creatinine and ejection fraction as the two most relevant features, we then build the machine learning survival prediction models on these two factors alone. Results: Our results of these two-feature models show not only that serum creatinine and ejection fraction are sufficient to predict survival of heart failure patients from medical records, but also that using these two features alone can lead to more accurate predictions than using the original dataset features in its entirety. We also carry out an analysis including the follow-up month of each patient: even in this case, serum creatinine and ejection fraction are the most predictive clinical features of the dataset, and are sufficient to predict patients’ survival. Conclusions: This discovery has the potential to impact on clinical practice, becoming a new supporting tool for physicians when predicting if a heart failure patient will survive or not. Indeed, medical doctors aiming at understanding if a patient will survive after heart failure may focus mainly on serum creatinine and ejection fraction. Keywords: Cardiovascular heart diseases, Heart failure, Serum creatinine, Ejection fraction, Medical records, Feature ranking, Feature selection, Biostatistics, Machine learning, Data mining, Biomedical informatics Background Cardiovascular diseases (CVDs) are disorders of the heart and blood vessels including, coronary heart disease (heart attacks), cerebrovascular diseases (strokes), heart failure (HF), and other types of pathology [1]. Altogether, car- diovascular diseases cause the death of approximately 17 *Correspondence: [email protected] 1Krembil Research Institute, Toronto, Ontario, Canada Full list of author information is available at the end of the article million people worldwide annually, with fatalities figures on the rise for first time in 50 years the United Kingdom [2]. In particular, heart failure occurs when the heart is unable to pump enough blood to the body, and it is usu- ally caused by diabetes, high blood pressure, or other heart conditions or diseases [3]. The clinical community groups heart failure into two types based on the ejection fraction value, that is the pro- portion of blood pumped out of the heart during a single © The Author(s). 2020 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. http://crossmark.crossref.org/dialog/?doi=10.1186/s12911-020-1023-5&domain=pdf http://orcid.org/0000-0001-9655-7142 http://orcid.org/0000-0002-2705-5728 mailto: [email protected] http://creativecommons.org/licenses/by/4.0/ http://creativecommons.org/publicdomain/zero/1.0/ Chicco and Jurman BMC Medical Informatics and Decision Making (2020) 20:16 Page 2 of 16 contraction, given as a percentage with physiological val- ues ranging between 50% and 75%. The former is heart failure due to reduced ejection fraction (HFrEF), previ- ously known as heart failure due to left ventricular (LV) systolic dysfunction or systolic heart failure and char- acterized by an ejection fraction smaller than 40% [4]. The latter is heart failure with preserved ejection frac- tion (HFpEF), formerly called diastolic heart failure or heart failure with normal ejection fraction. In this case, the left ventricle contracts normally during systole, but the ventricle is stiff and fails to relax normally during diastole, thus impairing filling [5–10]. For the quantitative evaluation of the disease pro- gression, clinicians rely on the New York Heart Asso- ciation (NYHA) functional classification, including four classes ranging from no symptoms from ordinary activi- ties (Class I) to a stage where any physical activity brings on discomfort and symptoms occur at rest (Class IV). Despite its widespread use, there is no consistent method of assessing the NYHA score, and this classification fails to reliably predict basic features, such as walking distance or exercise tolerance on formal testing [11]. Given the importance of a vital organ such as the heart, predicting heart failure has become a priority for medi- cal doctors and physicians, but to date forecasting heart failure-related events in clinical practice usually has failed to reach high accuracy [12]. In this context, electronic health records (EHRs, also called medical records) can be considered a useful resource of information to unveil hidden and non-obvious correlations and relationships between patients’ data, not only for research but also for clinical practice [13, 14] and for debunking traditional myths on risk factors [15, 16]. To this aim, several screening studies have been conducted in the last years, covering different conditions and demo- graphics and with different data sources, to deepen the knowledge on the risk factors. Among them, it is worth mentioning the PLIC study [17], where EHRs, blood test, single-nucleotide polymorphisms (SNPs), carotid ultra- sound imaging, and metagenomics data have been col- lected in a four-visit longitudinal screening throughout 15 years in Milan (Italy, EU) to support a better assessment of cardiovascular disease risk. Machine learning applied to medical records, in partic- ular, can be an effective tool both to predict the survival of each patient having heart failure symptoms [18, 19], and to detect the most important clinical features (or risk factors) that may lead to heart failure [20, 21]. Scien- tists can take advantage of machine learning not only for clinical prediction [22, 23], but also for feature ranking [24]. Computational intelligence, especially, shows its pre- dictive power when applied to medical records [25, 26], or coupled with imaging [27–29]. Further, deep learning and meta-analysis studies applied to this field have also recently appeared in the literature [30–33], improving on human specialists’ performance [34], albeit showing lower accuracy (0.75 versus 0.59). Modeling survival for heart failure (and CVDs in gen- eral) is still a problem nowadays, both in terms of achiev- ing high prediction accuracy and identifying the driving factors. Most of the models developed for this purpose reach only modest accuracy [35], with limited inter- pretability from the predicting variables [36]. More recent models show improvements, especially if the survival out- come is coupled with additional targets (for example, hospitalization [37]). Although scientists have identified a broad set of predictors and indicators, there is no shared consensus on their relative impact on survival prediction [38]. As pointed out by Sakamoto and colleagues [39], this situation is largely due to a lack of reproducibility, which prevents drawing definitive conclusions about the impor- tance of the detected factors. Further, this lack of repro- ducibility strongly affects model performances: general- ization to external validation datasets is often inconsistent and achieves only modest discrimination. Consequently, risk scores distilled from the models suffer similar prob- lems, limiting their reliability [40]. Such uncertainty has led to the proliferation of new risk scores appearing in the literature in the last years, with mixed results [41–47]. As a partial solution to improve models’ effectiveness, recent published studies included cohorts restricted to specific classes of patients (for example, elderly or dia- betic) [48, 49]. These attempts have led to tailored models and risk scores [50, 51] with better but still not optimal performance. In this paper, we analyze a dataset of medical records of patients having heart failure released by Ahmad and colleagues [52] in July 2017. Ahmad and colleagues [52] employed traditional biostatistics time-dependent mod- els (such as Cox regression [53] and Kaplan–Meier sur- vival plots [54]) to predict mortality and identify the key features of 299 Pakistan patients having heart failure, from their medical records. Together with their analy- sis description and results, Ahmad and coworkers made their dataset publicly available online (“Dataset” section), making it freely accessible to the scientific community [55]. Afterwards, Zahid and colleagues [56] analyzed the same dataset to elaborate two different sex-based mortal- ity prediction models: one for men and one for women. Although the two aforementioned studies [52, 56] pre- sented interesting results, they tackled the problem by standard biostatistics methods, leaving room for machine learning approaches. We aim here to fill this gap by using several data mining techniques first to predict survival of the patients, and then to rank the most important features included in the medical records. As major result, we show that the top predictive performances can be reached by machine learning methods with just two features, none Chicco and Jurman BMC Medical Informatics and Decision Making (2020) 20:16 Page 3 of 16 of them coming unexpected: one is ejection fraction, and the other is serum creatinine, well known in the literature as a major driver of heart failure [57–62], and also a key biomarker in renal dysfunction [63–65]. In particular, we first describe the analyzed dataset and its features (“Dataset” section), and then the methods we employed for survival prediction and feature ranking (“Methods” section). In the Results section (“Results” section), we report the survival pre- diction performances obtained through all the employed classifiers (“Survival machine learning prediction on all clinical features” section), the ranking of the features obtained through traditional biostatistics techniques and machine learning (“Feature ranking results” section), and the survival prediction performances achieved by employ- ing only the top two features identified through fea- ture ranking (ejection fraction and serum creatinine, “Survival machine learning prediction on serum creati- nine and ejection fraction alone” section). Later, we report and describe the results of the analysis that includes the patients’ follow-up time (Table 11). Finally, we discuss the results (“Discussion” section) and draw some conclusions at the end of the manuscript (“Conclusions” section). Dataset We analyzed a dataset containing the medical records of 299 heart failure patients collected at the Faisalabad Insti- tute of Cardiology and at the Allied Hospital in Faisalabad (Punjab, Pakistan), during April–December 2015 [52, 66]. The patients consisted of 105 women and 194 men, and their ages range between 40 and 95 years old (Table 1). All 299 patients had left ventricular systolic dysfunction and had previous heart failures that put them in classes III or IV of New York Heart Association (NYHA) classification of the stages of heart failure [67]. The dataset contains 13 features, which report clinical, body, and lifestyle information (Table 1), that we briefly describe here. Some features are binary: anaemia, high blood pressure, diabetes, sex, and smoking (Table 1). The hospital physician considered a patient having anaemia if haematocrit levels were lower than 36% [52]. Unfortu- nately, the original dataset manuscript provides no defini- tion of high blood pressure [52]. Regarding the features, the creatinine phosphokinase (CPK) states the level of the CPK enzyme in blood. When a muscle tissue gets damaged, CPK flows into the blood. Therefore, high levels of CPK in the blood of a patient might indicate a heart failure or injury [68]. The ejec- tion fraction states the percentage of how much blood the left ventricle pumps out with each contraction. The serum creatinine is a waste product generated by cre- atine, when a muscle breaks down. Especially, doctors focus on serum creatinine in blood to check kidney func- tion. If a patient has high levels of serum creatinine, it may indicate renal dysfunction [69]. Sodium is a min- eral that serves for the correct functioning of muscles and nerves. The serum sodium test is a routine blood exam that indicates if a patient has normal levels of sodium in the blood. An abnormally low level of sodium in the blood might be caused by heart failure [70]. The death event feature, that we use as the target in our binary classifica- tion study, states if the patient died or survived before the Table 1 Meanings, measurement units, and intervals of each feature of the dataset Feature Explanation Measurement Range Age Age of the patient Years [40, ..., 95] Anaemia Decrease of red blood cells or hemoglobin Boolean 0, 1 High blood pressure If a patient has hypertension Boolean 0, 1 Creatinine phosphokinase Level of the CPK enzyme in the blood mcg/L [23, ..., 7861] (CPK) Diabetes If the patient has diabetes Boolean 0, 1 Ejection fraction Percentage of blood leaving Percentage [14, ..., 80] the heart at each contraction Sex Woman or man Binary 0, 1 Platelets Platelets in the blood kiloplatelets/mL [25.01, ..., 850.00] Serum creatinine Level of creatinine in the blood mg/dL [0.50, ..., 9.40] Serum sodium Level of sodium in the blood mEq/L [114, ..., 148] Smoking If the patient smokes Boolean 0, 1 Time Follow-up period Days [4,...,285] (target) death event If the patient died during the follow-up period Boolean 0, 1 mcg/L: micrograms per liter. mL: microliter. mEq/L: milliequivalents per litre Chicco and Jurman BMC Medical Informatics and Decision Making (2020) 20:16 Page 4 of 16 Table 2 Statistical quantitative description of the category features Full sample Dead patients Survived patients Category feature # % # % # % Anaemia (0: false) 170 56.86 50 52.08 120 59.11 Anaemia (1: true) 129 43.14 46 47.92 3 40.89 High blood pressure (0: false) 194 64.88 57 59.38 137 67.49 High blood pressure (1: true) 105 35.12 39 40.62 66 32.51 Diabetes (0: false) 174 58.19 56 58.33 118 58.13 Diabetes (1: true) 125 41.81 40 41.67 85 41.87 Sex (0: woman) 105 35.12 34 35.42 71 34.98 Sex (1: man) 194 64.88 62 64.58 132 65.02 Smoking (0: false) 203 67.89 66 68.75 137 67.49 Smoking (1: true) 96 32.11 30 31.25 66 32.51 #: number of patients. %: percentage of patients. Full sample: 299 individuals. Dead patients: 96 individuals. Survived patients: 203 individuals. end of the follow-up period, that was 130 days on aver- age [52]. The original dataset article [52] unfortunately does not indicate if any patient had primary kidney dis- ease, and provides no additional information about what type of follow-up was carried out. Regarding the dataset imbalance, the survived patients (death event = 0) are 203, while the dead patients (death event = 1) are 96. In statistical terms, there are 32.11% positives and 67.89% negatives. As done by the original data curators [52], we repre- sented this dataset as a table having 299 rows (patients) and 13 columns (features). For clarification purposes, we slightly changed the names of some features of the origi- nal dataset (Additional file 1). We report the quantitative characteristics of the dataset in Table 2 and Table 3. Addi- tional information about this dataset can be found in the original dataset curators publication [52, 66]. Methods In this section, we first list the machine learning methods we used for the binary classification of the survival (“Sur- vival prediction classifiers” section), and the biostatistics and machine learning methods we employed for the fea- ture ranking (“Feature ranking” section), discarding each patient’s follow-up time. We then describe the logistic regression algorithm we employed to predict survival and to perform the feature ranking as a function of the follow-up time (“Stratified logistic regression” section). We implemented all the methods with the open source R programming language, and made it publically freely available online (Data and software availability). Survival prediction classifiers This part of our analysis focuses on the binary prediction of the survival of the patients in the follow-up period. To predict patients survival, we employed ten dif- ferent methods from different machine learning areas. The classifiers include one linear statistical method (Lin- ear Regression [71]), three tree-based methods (Random Forests [72], One Rule [73], Decision Tree [74]), one Artificial Neural Network (perceptron [75]), two Support Vector Machines (linear, and with Gaussian radial ker- nel [76]), one instance-based learning model (k-Nearest Neighbors [77]), one probabilistic classifier (Naïve Bayes Table 3 Statistical quantitative description of the numeric features Full sample Dead patients Survived patients Numeric feature Median Mean σ Median Mean σ Median Mean σ Age 60.00 60.83 11.89 65.00 65.22 13.21 60.00 58.76 10.64 Creatinine phosphokinase 250.00 581.80 970.29 259.00 670.20 1316.58 245.00 540.10 753.80 Ejection fraction 38.00 38.08 11.83 30.00 33.47 12.53 38.00 40.27 10.86 Platelets 262.00 263.36 97.80 258.50 256.38 98.53 263.00 266.66 97.53 Serum creatinine 1.10 1.39 1.03 1.30 1.84 1.47 1.00 1.19 0.65 Serum sodium 137.00 136.60 4.41 135.50 135.40 5.00 137.00 137.20 3.98 Time 115.00 130.30 77.61 44.50 70.89 62.38 172.00 158.30 67.74 Full sample: 299 individuals. Dead patients: 96 individuals. Survived patients: 203 individuals. σ : standard deviation Chicco and Jurman BMC Medical Informatics and Decision Making (2020) 20:16 Page 5 of 16 [78]), and an ensemble boosting method (Gradient Boost- ing [79]). We measured the prediction results through common confusion matrix rates such as Matthews correlation coefficient (MCC) [80], receiver operating characteristic (ROC) area under the curve, and precision-recall (PR) area under the curve (Additional file 1) [81]. The MCC takes into account the dataset imbalance and generates a high score only if the predictor performed well both on the majority of negative data instances and on the major- ity of positive data instances [82–84]. Therefore, we give more importance to the MCC than to the other confusion matrix metrics, and rank the results based on the MCC. Feature ranking For the feature ranking, we employed a traditional univari- ate biostatistics analysis followed by a machine learning analysis; afterwards, we compared the results of the two approaches. Biostatistics. We used common univariate tests such as Mann–Whitney U test [85], Pearson correlation coeffi- cient [86], and chi square test [87] to compare the distri- bution of each feature between the two groups (survived individuals and dead patients), plus the Shapiro–Wilk test [88] to check the distribution of each feature. Each test has a different meaning but all of them produce a score (a coefficient for the PCC, and a p-value for the other tests) representing the likelihood of a feature to be asso- ciated to the target. These scores can then be employed to produce a ranking, that lists the features from the most target-related to the least target-related. The Mann–Whitney U test (or Wilcoxon rank–sum test) [85], applied to each feature in relation to the death event target, detects whether we can reject the null hypothesis that the distribution of the each feature for the groups of samples defined by death event are the same. A low p-value of this test (close to 0) means that the ana- lyzed feature strongly relates to death event, while a high p-value (close to 1) means the opposite. The Pearson cor- relation coefficient (or Pearson product-moment correla- tion coefficient, PCC) [86] indicates the linear correlation between elements of two lists, showing the same elements on different positions. The absolute value of PCC gener- ates a high value (close to 1) if the elements of the two lists have linear correlation, and a low value (close to 0) otherwise. The chi square test (or χ2 test) [87] between two fea- tures checks how likely an observed distribution is due to chance [89]. A low p-value (close to 0) means that the two features have a strong relation; a high p-value (close to 1) means, instead, that the null hypothesis of independence cannot be discarded. Similar to what Miguel and colleagues did on a breast cancer dataset [90], we decided also to take advantage of the Shapiro–Wilk test [88] to assess if each feature was extracted from a normal distribution. Machine learning. Regarding machine learning feature ranking, we focused only on Random Forests [72, 91], because as it turned out to be the top performing clas- sifier on the complete dataset (“Feature ranking results” section). Random Forests [72] provides two feature rank- ing techniques: mean accuracy reduction and Gini impu- rity reduction [92]. During training, Random Forests gen- erates several random Decision Trees that it applies to data subsets, containing a subsets both of data instances and of features. In the end, Random Forests checks all the binary outcomes of these decisions trees and chooses its final outcome through a majority vote. The feature rank- ing based upon the mean accuracy decreases counts how much the prediction accuracy decreases, when a partic- ular feature is removed. The method then compares this accuracy with the accuracy obtained by using all the fea- tures, and considers this difference as the importance of that specific feature: the larger the accuracy drop, the more important the feature. The other feature ranking method works similarly, but is based upon the Gini impu- rity decrease [91]: the more the Gini impurity drops, the more important the feature. Aggregate feature rankings and prediction on the top features Starting from the whole dataset D we generated a col- lection D = {{Dtri , Dtsi }}N i=1 of N Monte Carlo stratified training/test partitions D = Dtri ∪ Dtsi with ratio 70%/30%. For each execution, we randomly selected 70% of patients for the training set, and used the remaining 30% for the test set. To make our predictions more realistic, we avoided using the same balance ratio of the whole complete dataset (32.11% positives and 67.89% negatives). This way, we had different balance ratios for each of the 100 executions with, on average, 32.06% positives and 66.94% negatives on average in the training sets, and with, on average, 32.22% positives and 67.78% negatives on average in the test sets. On the N training portions Dtr1 , . . . , D tr N we applied seven different feature ranking methods, namely RRe- liefF [93–95], Max-Min Parents and Children [96–98], Random Forest [72], One Rule [73], Recursive Partition- ing and Regression Trees [99], Support Vector Machines with linear kernel [100] and eXtreme Gradient Boosting [79, 101, 102], using the feature death event as the tar- get and obtaining 7N ranked lists of the 11 features. Agglomerating all the 7N features into the single Borda list [103, 104] we obtained the global list (Fig. 2 for N = 100), together with the Borda count score of each fea- ture, corresponding to the average position across all 7N lists, and thus the lower the score, the more important the feature. Chicco and Jurman BMC Medical Informatics and Decision Making (2020) 20:16 Page 6 of 16 We then used only the top–two features, namely serum creatinine and ejection fraction to build on each sub- set Dtri three classifiers, namely Random Forests (RF), Support Vector Machine with Gaussian Kernel (GSVM) and eXtreme Gradient Boosting (XGB). Finally, we then applied the trained models to the corresponding test por- tions Dtsi with the aforementioned top–2 features and averaged the obtained performances modelwise on the N test set instances. For the feature ranking and the classification made on the top two features, we employed different sets of the machine learning methods than the ones we used for the survival prediction on the complete dataset (“Survival prediction classifiers” section): RReliefF, Max- Min Parents and Children, Random Forests, One Rule, Recursive Partitioning and Regression Trees Support Vec- tor Machines with linear kernel, and eXtreme Gradient Boosting, for the feature ranking, and Random Forests, Gradient Boosting, and SVM with radial kernel. We decided to use three different sets of methods because we aimed to demonstrate the generalisability of our approach, by showing that our computational solution is not only valid with few machine learning classifiers, but rather works for several groups of methods. Regarding the final prediction using only the top two selected features, we chose Random Forests because it resulted in being the top performing classifier on the complete feature dataset (“Survival machine learn- ing prediction on all clinical features” section) and it is universally considered an efficient method for fea- ture ranking [92]. We then chose Gradient Boosting and Support Vector Machine with radial Gaussian kernel because both these methods have shown efficient per- formances in feature ranking with medical informatics data [105, 106]. Stratified logistic regression In the just-described first analysis, we wanted to predict the survival of patients and to detect the clinical feature importance in the follow-up time, without considering its different extent for each patient. In the second analysis, we decided to include the follow-up time, to see if the survival prediction results or the feature ranking results would change. To analyze this aspect, we mapped the orig- inal dataset time feature (containing the days of follow-up) into a month variable, where month 0 means that fewer than 30 days have gone by, month 1 means between 30 and 60 days, month 2 means between 60 and 90 days, and so on. We then applied a stratified logistic regression [107] to the complete dataset, including all the original clini- cal features and the derived follow-up month feature. We measured the prediction with the aforementioned confu- sion matrix metrics (MCC, F1 score, etc.), and the feature ranking importance as the logistic regression model coef- ficient for each variable. Results In this section, we first describe the results we obtained for the survival prediction on the complete dataset (“Survival machine learning prediction on all clinical fea- tures” section), the results obtained for the feature rank- ing (“Feature ranking results” section), and the results on the survival prediction when using only the top two most important features of the dataset (“Survival machine learning prediction on serum creatinine and ejection frac- tion alone” section and “Serum creatinine and ejection fraction linear separability” section), all independently from the follow-up time. We then report and discuss the results achieved by including the follow-up time of each patient in the survival prediction and feature rank- ing (“Survival prediction and feature ranking including the follow-up period” section). Survival machine learning prediction on all clinical features We employed several methods to predict the survival of the patients. We applied each method 100 times and reported the mean result score (Table 4). For methods that needed hyper-parameter optimization (neural network, Support Vector Machine, and k-Nearest Neighbors), we split the dataset into 60% (179 randomly selected patients) for the training set, 20% (60 randomly selected patients) for the validation set, and 20% (the remaining 60 patients) for the test set. To choose the top hyper-parameters, we used a grid search and selected the models that generated the highest Matthews correlation coefficient [83]. For the other methods …