I try to respond without delay usually. While I think one or two days is an acceptable timeframe to reply to a text message or email, it's rude to not reply for a week or more. Once my younger son was invited to the birthday party of his friend from school -- the invitation given on a Monday for the coming Saturday for which I RSVP'd right away, but on Saturday while at the party we found out that we were the only ones who attended the party from his classroom and that no one else even bothered to RSVP no. The birthday girl's mom was upset over this lack of communication. I see this happening always, especially to reply with a No.
But as you pointed out people communicate differently. Sometimes a silence means a loud no. :)