While this homemade paint recipe is edible, it doesn't taste good and I don't recommend ingesting it. If the baby licks it off his finger, though, no worries.
Store any leftover finger paint in an airtight container, however, for best results, use immediately.