But keep in mind that you will also want to consider whether or not the sacraments are being rightly administered.
I understand if this sounds a little Roman Catholic to you. Evangelicals today don't typically think a lot about the Lord's Supper and Baptism or consider them to be very important. Just hang with me for a second, and I think you will understand why this is so.
The sacraments are another form of gospel ministry. They are visual sermons, so to speak. They help to confirm you in the truth of God's promise and build you up in the faith, just like the reading and preaching of God's word.
You might say that baptism and the Lord's Supper are extensions of the preaching of the gospel. They compliment and assist it by doing to the eyes what preaching does to the ears.
Admittedly, its nothing flashy. There are no light shows or glitzy showmanship. The sacraments contain common elements and are presented in a very plain fashion. That's how God normally works though, and the power is in the plain-ness.
For instance, in baptism you see what God does when one first comes to faith. As you behold the waters flow over that person, you can remember that "He saved us, not because of works done by us in righteousness, but according to his own mercy, by the washing of regeneration and renewal of the Holy Spirit." Tit 3:5 ESV
When you come to the communion table the sufferings of Christ are put before your eyes (and even put into your hands!). Thus, you have a powerful testimony to the atonement that was made for your sins.
That is why you should seek a church that takes baptism seriously and is committed to having Communion on a regular basis. These are ordinances that God has instituted for your spiritual well-being. If a church isn't committed to them (or if a church is administering them in a way that is not in accord with Scripture) that church isn't committed to your nurture in the gospel.
In the future, we'll likely do a series on the sacraments. There is a lot to learn on this subject! For now, it is enough to say that the sacraments should be on your radar when looking around for a church.