It's a shitty and awkward sentence to begin with (with which to begin), primarily because it unnecessarily uses the passive voice to entangle two ideas that are clearer when approached one at a time in the active voice:
I didn't want anyone to read out of that book to me. Why did you want to bring it up?
See? Contrary to what is suggested above, the whole thing is MUCH better with the active voice. Note that in the case of "bring up," "up" is an adverb, not a preposition, so I am not ending the second sentence with a preposition.