The First Amendment specifically speaks about government not limiting free expression. An indeed, school boards are a branch of the government, not a private organization. Their actions may be seen as a real infraction on the First Amendment.
If the state doesn't limit freedom of expression by choosing what material to teach in schools (which it does) then it doesn't limit it by choosing what material to host in school libraries (which it does).
If you want to say removing these books from school libraries is an illegitimate constraint on freedom of expression, then so is the school curriculum. So is public education generally.
While removing a book from a school library by the school board may be a sensible act, and does not violate the letter of the First Amendment ("Congress shall make no law..."), it definitely has something to do with the spirit of it, that it, the interaction of government and free speech. It's very certainly something to keep an eye on.
Thank you for the correction.