Many libraries offer library cards to very young children and even babies. In the early 1990s a number of libraries gave out packets of information about early literacy to new mothers. These packets often included applications for library cards. Library cards for the babies themselves came soon after. Libraries often issue the card in conjunction with early literacy development programs. The cards are designed to get parents to start thinking about developing their child's reading skills as early as possible--and to get them into the library!
See the links below for examples of libraries that offer these services.