The Lord says to Peter: “I say to you,” He says, “that you are Peter, and upon this rock I will build my Church, and the gates of hell will not overcome it. And to you I will give the keys of the kingdom of heaven: and whatever things you bind on earth shall be bound also in heaven, and whatever you loose on earth, they shall be loosed also in heaven” [Matt. 16:18-19]. And again He says to him after His resurrection: “Feed my sheep” [John 21:17]. On him He builds the Church, and to him He gives the command to feed the sheep; and although He assigns a like power to all the Apostles, yet He founded a single chair, and He established by His own authority a source and an intrinsic reason for that unity. Indeed, the others were that also which Peter was; but a primacy is given to Peter, whereby it is made clear that there is but one Church and one chair. So too, all are shepherds, and the flock is shown to be one, fed by all the Apostles in single-minded accord. If someone does not hold fast to this unity of Peter, can he imagine that he still holds the faith? If he desert the chair of Peter upon whom the Church was built, can he still be confident that he is in the Church? (“Commentaries on Twelve of David’s Psalms”)

He [Peter], then, who before was silent, to teach us that we ought not to repeat the words of the impious, this one, I say, when he heard: “But who do you say I am” [Matt. 16:15], immediately, not unmindful of his station, exercised his primacy, that is, the primacy of confession, not of honor; the primacy of belief, not of rank. That is to say: “Now let no one outdo me; now is my role; I ought to compensate for my silence; the fact that I was silent ought to be of benefit. My tongue does not have perplexities; faith should come forth without difficulty. While some were casting forth filth, although the filth of another’s impiety but proclaimed by them, who said that Christ was either Elias, or Jeremias, or one of the prophets; for that voice had filth, that voice had perplexities; while some, I say, are washing away this filth, while in some these perplexities are being eradicated, let our voice resound that Christ is the Son of God. My words are pure, in which expressed impiety has left no perplexities.”
This, then, is Peter, who has replied for the rest of the Apostles; rather, before the rest of men. And so he is called the foundation, because he knows how to preserve not only his own but the common foundation. Christ agreed with him; the Father revealed it to him. For he who speaks of the true generation of the Father, received it from the Father, did not receive it from the flesh [cf. Matt. 16:16-17].
Faith, then, is the foundation of the Church, for it was not said of Peter’s flesh, but of his faith, that “the gates of hell shall not prevail against it” [Matt. 16:18]. But his confession of faith conquered hell. And this confession did not shut out one heresy, for, since the Church like a good ship is often buffeted by many waves, the foundation of the Church should prevail against all heresies.
The day will fail me sooner than the names of heretics and the different sects, yet against all is this general faith – that Christ is the Son of God, and eternal from the Father, and born of the Virgin Mary. (“The Sacrament of the Incarnation of Our Lord,” 32-35, The Fathers of the Church, Vol. 44 [Washington: The Catholic University of America Press, 1963], pp. 230-31)

Jesus said to them: Who do men say that I am? Simon Peter answering said, The Christ of God (Lk. ix.20). If it is enough for Paul ‘to know nothing but Christ Jesus and Him crucified,’ (1 Cor. ii.2), what more is to be desired by me than to know Christ? For in this one name is the expression of His Divinity and Incarnation, and faith in His Passion. And accordingly though the other apostles knew, yet Peter answers before the rest, ‘Thou art the Christ the Son of God’... Believe, therefore, as Peter believed, that thou also mayest be blessed, and that thou also mayest deserve to hear, ‘Because flesh and blood hath not revealed it to thee, but My Father who is in heaven’... Peter therefore did not wait for the opinion of the people, but produced his own, saying, ‘Thou art the Christ the Son of the living God’: Who ever is, began not to be, nor ceases to be. Great is the grace of Christ, who has imparted almost all His own names to His disciples. ‘I am,’ said He, ‘the light of the world,’ and yet with that very name in which He glories, He favored His disciples, saying, ‘Ye are the light of the world.’ ‘I am the living bread’; and ‘we all are one bread’ (1 Cor. x.17)... Christ is the rock, for ‘they drank of the same spiritual rock that followed them, and the rock was Christ’ (1 Cor. x.4); also He denied not to His disciple the grace of this name; that he should be Peter, because he has from the rock (petra) the solidity of constancy, the firmness of faith. Make an effort, therefore, to be a rock! Do not seek the rock outside of yourself, but within yourself! Your rock is your deed, your rock is your mind. Upon this rock your house is built. Your rock is your faith, and faith is the foundation of the Church. If you are a rock, you will be in the Church, because the Church is on a rock. If you are in the Church the gates of hell will not prevail against you…He who has conquered the flesh is a foundation of the Church; and if he cannot equal Peter, he can imitate him. (Commentary in Luke VI.98, CSEL 32.4)