First Nephi 7:15

Now behold, I say unto you that if ye will return unto Jerusalem ye shall also perish with them. And now, if ye have choice, go up to the land, and remember the words which I speak unto you, that if ye go ye will also perish; for thus the Spirit of the Lord constraineth me that I should speak.

We always have our agency, but we do not get to choose the consequences.