Yes, a college can have both division 1 and division 3 sports (although not in the same sport, i.e. a school won't have a division I football team and a division III football team). Here are a few exmples: Colorado College has division 1 mens hockey and womens soccer and is division 3 for the rest, Johns Hopkins has division 1 mens and womens lacrosse programs and is division 3 for the rest, Middlebury has a division 1 skiing program and is division 3 for the rest.
For a number of reasons, there are not very many schools that do have both division 1 and 3 programs. One of the reasons that it's not altogether common because the majority of division 1 programs in any sport are schools who, because of their size, are prevented by NCAA bylaws from competing in division 3. Division 3 schools tend to be small schools, it would be unfair for a school like say the University of Michigan with 40,000+ students to be allowed to compete in a league intended for schools with a student body of fewer than say 5,000. Another factor that prevents some schools (nowadays) from having both division 1 and 3 programs is Title IX, which mandates equal interscholastic athletic opportunities for both men and women. This means that there must be equal funding for things like scholarships between mens and womens sports at any given school, and division 3 sports cannot give scholarships so if you have a division 1 mens team in any sport that has scholarships than there must be a corresponding womens sport. For example, at Colorado College it used to be just mens ice hockey that was division 1, but when Title IX came into effect in the '70's Colorado College had to either a) drop from ice hockey from division 1 to division 3, b) keep ice hockey in division 1 but stop offering athletic scholarships for it, or c) elevate a womens team from division 3 to division 1. Colorado College chose option c, and since they did not have a womens ice hockey team they chose to elevate their at-the-time division 3 womens soccer team to division 1. Lots of schools in similar positions chose to simply go division 3 across the board and not have to worry about making sure that athletic scholarships were equal between the sexes.
What it boils down to is that yes, it is possible for a school to be division 1 in some sports but division 3 in others. However, there are only a handful of schools in the country that fall into that category.
As far as division 3 teams competing against division 1 teams, there's nothing stopping them from scrimmaging each other. I played division 3 lacrosse in college, and every year we would have a couple of exhibition games against a few division 1 teams that were close by. As far as competing in a contest that actually counts towards rankings or league standings, no that doesn't happen. Division 1 and division 3 are 2 completely separate leagues, it'd be like a team from the NFL playing against a team from the Canadian Football League and having it count towards the standings. Division 3 programs are smaller programs that cannot compete recruiting-wise with larger division 1 schools, particularly considering the aforementioned fact that division 1 programs can offer athletic scholarships while division 3 programs cannot. Add in the fact that division 1 rules allow for teams to basically practice year round but division 3 programs prohibit official, mandated team activities outside of the season and it becomes pretty clear how much of an edge a division 1 program would have over a division 3 program. When we would scrimmage against division 1 teams, they weren't even particularly good division 1 teams and we still didn't stand a chance. It was fun, but having it actually count for anything would have been INCREDIBLY unfair to us