A chance for me to put on my Trained-As-A-Linguist Hat...

IsaoShi's right: "slowly walked" is not an infinitive phrase. Any prohbition relating to infinitives would not apply to this phrase since it's a finite phrase in the perfect (simple past) tense.

And DPH is right: the "don't split infinitives" (i.e., don't place anything between a "to" and the bare infinitive form of a verb) is a made-up rule intending to copy Latin and has no place being applied to English.

However, if the text did actually contain a split infinitive, it might be advisable to remove it to help the text to also capture ( <== that's a split infinite right there) the mood of the period being depicted. Read original literature from around about the time of the 18th-century-turning-into-the-19th and you will notice that there are very few (if any) split infinitives. The first time I read Frankenstein, the lack of split infinitives and where it appears Mary Shelley deliberately phrased things to avoid the split infinitives stood out.