It can be a challenge to figure out what to wear to family photoshoots. We have the same challenge! James and I have a son and a daughter, so coordinating all four of us can be tricky. First and foremost, everyone should be unique in some way. You do not need to wear matching outfits. Each outfit should also fit the person's personality. Remember that these photos are meant to represent you and your family. So, if someone never wears a dress, don't wear a dress to your family portraits. If someone hates a certain color, they shouldn't wear that color. Basically, be true to who you are. That being said, there are ways to have everyone looking their best while still blending your outfits.
You can choose an exact shade (like navy), or you can do monochromatic coordinating. So, someone wears royal blue, someone wears navy blue, someone has sky blue, etc. This doesn't mean everyone wears blue from head to toe. Part of each person's outfit could be blue. Or, in the case of the family above, they coordinated blue and white.
We've all seen those photos of everyone in white t-shirts and blue jeans. Personally, I don't love this because it means everyone matches exactly. Instead, I've adapted this to simply be a white top. The top doesn't match everyone else, so it's a way to have your personality show through a bit.
You can also coordinate textures or patterns instead of the actual colors. Everyone can wear sweaters, like the photo above. You can all wear plaids, or a certain type of pattern. And if everyone wears jeans, this will coordinate it even more.
You can find which colors compliment others based on the Color Wheel. There are a few ways to use this. You can select all cool colors (like the photo above: blues, greens, purples), or you could use complementary colors (like blue and orange), etc. If you look it up on Pinterest, you'll find several different color schemes based on the Color Wheel.