If they say that all rights are granted, then you can use them. Period. If it is for non-commercial use only and it is a free comic, then you can use it. A comic is no different than other artwork, there's just more of it. As a courtesy, you should give credits, but unless the readme requires it, then you are not obligated to do it.