|
workape posted:
My question based off your comments, what is the exact FEX limit on the 5k series switches? I've seen mixed numbers using my google foo for 12 or 16 FEX's max depending on the model. Honestly my googlefoo is sucking today so I haven't been able to find a straight answer Does anyone know the max FEX count for a 5596UP and a 5548UP switch? I'm curious to know myself.
|
#
?
Oct 21, 2011 21:33
|
|
|
|
| # ? May 18, 2024 03:21 |
|
|
Langolas posted:My question based off your comments, what is the exact FEX limit on the 5k series switches? I've seen mixed numbers using my google foo for 12 or 16 FEX's max depending on the model. Honestly my googlefoo is sucking today so I haven't been able to find a straight answer According to Cisco, it appears that the 5500 series has a limit of 16 FEX units, and that is reduced to 8 when you are using the layer 3 daughter card. Although I'm sure there is a Cisco way to upgrade that (buy a 7k series). Your Google-fu doesn't suck, you just had to to an search for "cisco nexus configuration limits". Obvious, right? 
|
|
#
?
Oct 21, 2011 23:06
|
|
|
BelDin posted:According to Cisco, it appears that the 5500 series has a limit of 16 FEX units, and that is reduced to 8 when you are using the layer 3 daughter card. Although I'm sure there is a Cisco way to upgrade that (buy a 7k series). Lol Thanks! And you gave me a link to Cisco itself I can give to my boss. Hes gonna be even more sad since we are gonna use the layer 3 daughtercard. Limited to 8 is gonna blow for what we were planning and the quote for a 7k our sales rep gave us made us cringe
|
|
#
?
Oct 22, 2011 17:47
|
|
|
Langolas posted:Lol Thanks! And you gave me a link to Cisco itself I can give to my boss. Hes gonna be even more sad since we are gonna use the layer 3 daughtercard. Limited to 8 is gonna blow for what we were planning and the quote for a 7k our sales rep gave us made us cringe This one shows the limits for the differences between the FEX maximum numbers conducting L2 and L3 operations. My current thinking is to get a few 3560s with 10G SFP+ modules and run HSRP/RSTP/SVI/EIGRP services on the pair to act as distribution switches until we can get budget for the Nexus 7ks. I've got others suggesting that we just plug them in our core router and treat it like a collapsed core, but I'd rather keep the core and distribution switches separated if possible. BelDin fucked around with this message at 18:41 on Oct 22, 2011 |
|
#
?
Oct 22, 2011 18:12
|
|
|
BelDin posted:This one shows the limits for the differences between the FEX maximum numbers conducting L2 and L3 operations. If you are going to be looking at the 7k's for the love of God make sure you review everything multiple times and get several SE's to look at the config. I got mine without issue, but a guy in town just fired their VAR because he trusted them to know what they were talking about and got burned so hard I thought he was going to be fired. His only saving grace was his excellently written RFP that clearly stated what he wanted and expected in plain enough english that the powers that be understood it.
|
|
#
?
Oct 23, 2011 03:16
|
|
|
workape posted:If you are going to be looking at the 7k's for the love of God make sure you review everything multiple times and get several SE's to look at the config. I got mine without issue, but a guy in town just fired their VAR because he trusted them to know what they were talking about and got burned so hard I thought he was going to be fired. His only saving grace was his excellently written RFP that clearly stated what he wanted and expected in plain enough english that the powers that be understood it. I see the 7ks to probably be a waste due to the small amount of traffic that will actually traverse our core. We are running 2x2GE portchannels (2 to each 3560) right now, and the most stressful item on that will be our end user file services, VDI infrastructure (latency sensitive), and upcoming VoIP pilot project. We should be able to make it work with the standard end-to-end QoS design without resorting to equipment of that size and cost. My personal driver for this is the reduction of oversubscription in the server room and convergence of our SAN using less equipment. When you are using 2960Gs for your primary SAN switch, you upgrade as soon as it is offered! I'm asking for any additional information in here for exactly the reason you just stated. I'm worried that I will buy the equipment and it won't do what I intend. No one told me about the 8 FEX L3 caveat, and came across it during my product research. I need this pair of equipment to scale across 14 racks, so that makes it an obvious no-go. It doesn't help matters that I have pressures to deliver a part list next week for what they need to outfit the new trailer, and my vendor rep. hasn't been very responsive this week. Kind of funny that after years of hobbling things together they are practically begging me to drop 150k on new network gear without making sure that the stuff will even do what we want it to do.
|
|
#
?
Oct 23, 2011 04:06
|
|
|
Strange problem. I have 3 vlans configured on two of our branch switches. I have IP addresses defined on 2 of those VLANs (this is an acquisition and the branch is on two completely seperate subnets, so I would like to be able to access it from both). As soon as I enable the second subnets VLAN interface, that subnets DHCP server fills up with BAD_ADDRESS entries, from a mac address that is the inverse of the IP in hex. If I disable the interfaces, it's fine and I do not see those entries. The interfaces do not get their IPs from DHCP either, they are statically set.
|
|
#
?
Oct 23, 2011 15:48
|
|
|
I'm not sure about the dhcp issue but layer two switches can only have one VLAN interface/IP address active at a time for management. Use routing to reach it from a different subnet.
|
|
#
?
Oct 23, 2011 16:00
|
|
|
falz posted:I'm not sure about the dhcp issue but layer two switches can only have one VLAN interface/IP address active at a time for management. Use routing to reach it from a different subnet. Anyway, I think I solved my problem. I had the voice vlan defined on the port the DHCP server was connected to. I just did a no switchport voice vlan on the interface and all is well now. Not even sure why it mattered, because they have shoretel phones which don't appear to utilize the voice vlan definition, they utilize a DHCP option to know which vlan to use.
|
|
#
?
Oct 23, 2011 16:20
|
|
|
What model of switch? Layer two only switches will auto shutdown an SVI when you add and enable a new one with an IP. Edit: looks like you're right. As of IOS 12.2 they allow it. I've just continued the 'single SVI' behavior from the XL days. falz fucked around with this message at 17:03 on Oct 23, 2011 |
|
#
?
Oct 23, 2011 16:44
|
|
|
so turns out nexus 5500's are absurdly expensive, and also I had a pair of dell 6248's lying around. As long as I stick to really simple 4 or 5 vlan and maybe one trunk over an lacp pair, will I be ok? Or are these things "extreme" level disasters? (those purple ones from the early days of gigabit)
|
|
#
?
Oct 23, 2011 23:50
|
|
|
I have ~20 Extreme switches deployed. They are pretty rock solid as basic L2/L3 switches.
|
|
#
?
Oct 24, 2011 01:04
|
|
|
Here is a picture to help: 192.168.1.0 ---- JUNIPER1 1.1.1.1 ------- 1.1.1.2 CISCO ASA 1.1.2.2 ------- 1.1.2.1 JUNIPER2 ----- 192.168.2.0 If I send a ping from 1.0 to 2.0 for instance, if JUNIPER1/JUNIPER2 both have default routes going to CISCO ASA, but CISCO ASA has NO routes, how does it know where to send a ping destined to the 2.0 network? Is this because the packet is encapsulated in the VPN tunnel and the destination changes from 2.0 to 1.1.2.1? Or maybe I am just not remembering to config right on the CISCO ASA and there were routes... Also, for those who helped me with the DNS stuff, thanks. It wasn't really for me so I gave what info I could to the person in need. Still working on it. EDIT: I think I just answered my own dumb question. In tunnel mode the public IP is what is used to route for destination. Disregard. Found it in my IINS book.
Bardlebee fucked around with this message at 10:04 on Oct 24, 2011 |
|
#
?
Oct 24, 2011 09:58
|
|
|
StabbinHobo posted:so turns out nexus 5500's are absurdly expensive, and also I had a pair of dell 6248's lying around. As long as I stick to really simple 4 or 5 vlan and maybe one trunk over an lacp pair, will I be ok? Or are these things "extreme" level disasters? (those purple ones from the early days of gigabit) How much were you expecting to pay and what's your discount rate?
|
|
#
?
Oct 24, 2011 16:48
|
|
|
5ks are expensive, but not compared to 7ks. One of the interesting things we just ran into, is that with UCS fabric interconnects (61xx, 62xx series boxes), and 7ks, neither the fabric interconnect nor the 7k can do san zoning. But, the 5k can, so if you want to go fcoe you'll, today, need 5ks to do the zoning. Which sucks. So we decided to simply preserve our investment in 9509s for the time being, and go native FC from the fabric interconnects to the 9509s.
|
|
#
?
Oct 24, 2011 17:36
|
|
|
I always forget Cisco MDS exists. I've never seen it in any deployment except at Cisco customer demos. I assume the product line has been around almost as long as 6500's have.
|
|
#
?
Oct 24, 2011 18:09
|
|
|
workape posted:How much were you expecting to pay and what's your discount rate? I figured they'd be a premium over 3560/3750's but not all the way up at 2x. The base platform isn't actually *that* bad from a price perspective, but the fleet of sfp's you have to buy for the ports to be usable add up fast. So anyone have disaster stories with Dell 6248s?
|
|
#
?
Oct 24, 2011 18:12
|
|
|
No experience with the 6248, but I sure hope they run a different code than the Dell M6220 blade switches. You do *not* want to get yourself into that trainwreck.
|
|
#
?
Oct 24, 2011 18:29
|
|
|
jwh posted:5ks are expensive, but not compared to 7ks. Have you worked with the 9148's? I'm looking at them for another project, but at the same time I am looking at putting an 8 port FC card into my 5500's and doing all of my zoning/etc out of them.
|
|
#
?
Oct 24, 2011 20:10
|
|
|
We have a bunch of 9148s in addition to our 9500s. I don't work with them directly, as we have storage networking folks that handle them, but they have a reputation (the 9148s) for being very nice boxes.
|
|
#
?
Oct 24, 2011 20:16
|
|
|
StabbinHobo posted:So anyone have disaster stories with Dell 6248s? jwh posted:I don't work with them directly, as we have storage networking folks that handle them, but they have a reputation (the 9148s) for being very nice boxes.
|
|
#
?
Oct 25, 2011 22:38
|
|
|
BelDin posted:I'm asking for any additional information in here for exactly the reason you just stated. I'm worried that I will buy the equipment and it won't do what I intend. No one told me about the 8 FEX L3 caveat, and came across it during my product research. I need this pair of equipment to scale across 14 racks, so that makes it an obvious no-go. Finally talked to the Cisco guy(s) brought in by a reseller: : Hi! I was wondering if you could give me some additional information on the Cisco Nexus line. (Goes into basics of the package selected, intended deployment, project background) Yes sir, that is the standard deployment with that product line. So, I was wondering... I need to connect the pair to our core, and I either need to use the L3 daughter card or buy a pair of switches to act as distribution with SVIs/VLANS. I noticed that the L3 card limits you to 8 FEX single homed per switch. Is this a hard limit or soft limit? I would love to be able to tie directly to our core and treat it as an access/distribution switch without additional equipment if it can be accomodated by buing extra licenses. I'm sorry sir... I'm not familiar with that aspect of the product. Let me get someone on the line more familiar with the product line.:*Cue waiting almost 10 minutes on hold waiting for the other Cisco guy*:*Repeat entire previous conversation, ask about L2/L3 issue* :I'm sorry sir, I'm afraid I don't know what you are looking at. :Here's the document ID of what I'm reading detailing the limits of the L3 daughter card.:Are you looking at a Cisco document? Is it in the release notes? I'm not seeing it in the release notes.:Yes, it is a document detailing the configuration limits for the product. Do you need the link for it on your site? :No, I found it... I didn't know the limit changed based on the use of that module. That's good to know! Thanks for the heads up! The conversation went downhill from there. I brought up flow control/caching/cut-through vs. store-and-forward switching regarding a 10Gb source and 1Gb/2Gb destination ports, FEX single-homed redundant connections (answer: 1 FEX is 1 FEX regardless of the number of connections to the same switch), and general vPC limitations. Oh, and did you know it supports LAN/SAN convergence when you are using iSCSI? FC/FCoE, now that's a different animal! I also asked if I was missing any questions that others typically ask, and got a somewhat abrupt "not really" before the Cisco guys ended their end of the call. For those of you using Nexus, is this also the level of support you have gotten from TAC when you've had issues with this line of equipment in production? I'd go back to stacked 3750x(s) if I thought it would give me a better support experience. I can understand that not everyone knows everything about all products, but you would think that a product line that is almost three years old would garner some level of knowledge there. I'm not exactly a beautiful and unique snowflake with my deployment here...
|
|
#
?
Oct 26, 2011 03:46
|
|
|
Much like with CRS-1 or 3's the Nexus line doesn't have near the depth of knowledge associated with it, as say the 6500. If you want hard hitting specific hardware questions like that you need to get to know your Cisco rep, and have him talk to a buddy of his with in the company. Remember Cisco is still a big company, and the Nexus line is a vastly different product, there are only X number of people who I would consider "competent" in the Nexus line, and they are in high demand. Cisco, unlike IOS, isn't monolithic and homogenous.
|
|
#
?
Oct 26, 2011 04:16
|
|
|
Like powercrazy said build up a relationship with your SE because they are really the guys that know how the poo poo works in practice and if not them then they know someone who does, and besides that it's literally their job to get you up to speed.
|
|
#
?
Oct 26, 2011 04:36
|
|
|
abigserve posted:Like powercrazy said build up a relationship with your SE because they are really the guys that know how the poo poo works in practice and if not them then they know someone who does, and besides that it's literally their job to get you up to speed. I've never had a relationship with a SE direct with Cisco. Typically, it's the old call up CDW or Insight and have them get someone on the phone. I guess I just always assumed that if I called up wanting SE expertise in the Nexus/MDS line of switches, they would have one that was somewhat familiar with that line of products. For example: The question I had about the switching types, I expected an answer along the lines of "Here's where to go to find the documentation for that aspect of the product line" or "I'll check on that and get back with you". I got, and didn't expect, the simple answer "It's got big buffers". The resulting questions I would have: So what happens when the buffer fills up because my equipment is streaming 10Gb iSCSI disk traffic to a 1Gb host server connection over 15 minutes? Is there a best practices document I should be following? Is there additional documentation I should be reading? Do I have to do anything special for effective deployment of the PFC or ECN features baked in? So is there a way to get a contact at Cisco independent of a reseller without having to buy direct from Cisco? We will be having to implement limited rollout of VoIP over the next six months, and I will have specific questions soon for planning and equipment purchasing reasons. I guess my problem is I dwell in the planning stages with a control freak mentality regarding any equipment touching our network and how it fits in the big picture.
|
|
#
?
Oct 26, 2011 05:09
|
|
|
What is the size and industry your company is in? If you do more than 50k/yr you'll have a specific rep, though you will of course be sharing him with many other businesses within your zipcode. If you are seriously looking at a full Nexus rollout, you will probably get Cisco's attention. Find a Cisco Gold Partner (there are a lot of them, and they advertise) and talk with them, find your rep, talk with him, then finally talk with the SE. The SE I promise you won't know very much about the Nexus line, but if he has been around for awhile, he will certainly know someone who does. Also don't use CDW, there are hundreds of cheaper and faster official and gray market resellers out there.
|
|
#
?
Oct 26, 2011 06:29
|
|
|
Powercrazy posted:What is the size and industry your company is in? If you do more than 50k/yr you'll have a specific rep, though you will of course be sharing him with many other businesses within your zipcode. If you are seriously looking at a full Nexus rollout, you will probably get Cisco's attention.
|
|
#
?
Oct 26, 2011 12:49
|
|
|
Powercrazy posted:What is the size and industry your company is in? If you do more than 50k/yr you'll have a specific rep, though you will of course be sharing him with many other businesses within your zipcode. If you are seriously looking at a full Nexus rollout, you will probably get Cisco's attention. We are a prime government contractor of about 150 people providing IT (among many other things) infrastructure support services to the main site contract of about 1600. This will probably go to about 2000 over the next year. The reason I typically get with CDWG is that they provide great budgetary estimates. The other guy's price is lower over half of the time.
|
|
#
?
Oct 26, 2011 14:54
|
|
|
There is a cisco account rep in your territory, just figure out who it is and get on the phone with them. They'll be able to bring in internal (Cisco) resources to talk to you. Ideally, your VAR should be doing this for you, also, but that's not always the case.
|
|
#
?
Oct 26, 2011 16:09
|
|
|
BelDin posted:We are a prime government contractor of about 150 people providing IT (among many other things) infrastructure support services to the main site contract of about 1600. This will probably go to about 2000 over the next year. There is a Cisco rep (with a team of SE's and others) assigned to the mission/agency you are supporting. It is just a matter of finding out who. If you have PM, shoot me one with your agency/mission and I can ask my Cisco rep if you want.
|
|
#
?
Oct 26, 2011 16:22
|
|
|
BelDin posted:We are a prime government contractor of about 150 people providing IT (among many other things) infrastructure support services to the main site contract of about 1600. This will probably go to about 2000 over the next year. Beeing a SE hopefully I can help you get in touch with the right people. Send me a PM or email (daniel@fnutt.net) if you want help.
|
|
#
?
Oct 26, 2011 17:03
|
|
|
routenull0 posted:There is a Cisco rep (with a team of SE's and others) assigned to the mission/agency you are supporting. It is just a matter of finding out who. If you have PM, shoot me one with your agency/mission and I can ask my Cisco rep if you want. See we have been working on deciding between a 5k or 7k with 5k rollout with our Cisco rep and hes been a great help. Our big problem is the white papers posted a little earlier in this thread talking about the 5k FEX limits with layer 3 cards is a deal breaker for us. That and not being able to do more than 96 etherchannel's on the 5ks. We were gonna dual home our servers and use 2k's to tie our racks in with 1gb lines. Now we probably are gonna have to go 7k's and have our 5ks be our datacenter 10gb boxes and run the 2ks off our 7k's to tie into everything. Of course we are still way up in the air with our design and the hardware we are getting, upgrading from a stack of 5 3750e's for our core to a 5k setup or a 7k setup is gonna be immense. Thankfully our Cisco rep here has a CCIE on staff to help us out and then a guy who has a ton of Cisco security experience for anything the CCIE doesn't know to well.
|
|
#
?
Oct 26, 2011 17:16
|
|
|
BelDin posted:For those of you using Nexus, is this also the level of support you have gotten from TAC when you've had issues with this line of equipment in production? I'd go back to stacked 3750x(s) if I thought it would give me a better support experience. TAC has been hit or miss on the 7k's for us. The 5k's are a different animal, since the 5500's have come out and their strong push for unified edge ports the support on that group has been excellent. The 7k's require a different skill set, and I would highly recommend getting involved in the communities that crop up in cities of people that deal with them. I learned about a vPC peer gateway issue from one of the guys here in St Louis before TAC even knew about it and he had a workaround that actually worked. It's since been fixed, but the community is where I've been getting most of my support as opposed to TAC.
|
|
#
?
Oct 26, 2011 18:14
|
|
|
workape posted:I would highly recommend getting involved in the communities that crop up in cities of people that deal with them. I learned about a vPC peer gateway issue from one of the guys here in St Louis before TAC even knew about it and he had a workaround that actually worked. Being that I am spending my next few months deploying a new DC in STL, this would be good info have if you can pass along.
|
|
#
?
Oct 26, 2011 21:53
|
|
|
routenull0 posted:Being that I am spending my next few months deploying a new DC in STL, this would be good info have if you can pass along. Little things like: If you are running Cisco NX-OS Release 5.1(1a) or Release 5.1(2) and you have the vPC peer-gateway feature enabled, you must upgrade both vPC peers to 5.1(4); otherwise the upgrade will be disruptive. The best one was we were pushed to get onto 5.1(2) as it would fix all of our problems and I got an outage window to do it and then the release was deferred literally less than a week after we got it installed. I am currently planning a 5.2 upgrade just after thanksgiving, mostly because I am a masochist and I can't get enough pain or upgrading my core infrastructure. Where in STL are you installing a DC?
|
|
#
?
Oct 26, 2011 22:45
|
|
|
Langolas posted:See we have been working on deciding between a 5k or 7k with 5k rollout with our Cisco rep and hes been a great help. Our big problem is the white papers posted a little earlier in this thread talking about the 5k FEX limits with layer 3 cards is a deal breaker for us. That and not being able to do more than 96 etherchannel's on the 5ks. We were gonna dual home our servers and use 2k's to tie our racks in with 1gb lines. Now we probably are gonna have to go 7k's and have our 5ks be our datacenter 10gb boxes and run the 2ks off our 7k's to tie into everything. Are you talking about dual homing the servers, or the FEX? If you're not planning on pushing more than 10Gb to the core and don't need L3 to the servers, you can try what I'm planning and get a pair of 3560s or better running IP Base and redundant power supplies for a EIGRP stub switch. You can get a good level of IP gateway reliability through HSRP, and your convergence time can be tweaked to acceptable for most use. Either that, or stack/chassis and portchannel to the core over a couple of switches. Unless you have an obscene amount of ACLs you should be able to push quite a bit of traffic with a smaller switch depending on the traffic levels you need to move. Our big goal was traffic increases in the server room, not out to the client network.
|
|
#
?
Oct 26, 2011 22:56
|
|
|
workape posted:Where in STL are you installing a DC? All government stuff. This site is all Juniper, but we had looked at Nexus 7009 and 7019s for another site to mirror, but ended up back with Juniper so the sites would remain exact. We first looked at Nexus before all the FAB2 stuff came out and it couldn't meet our requirements for density and speed. Now that's all changed with the announcements of FAB2 in the 7019s. A little to late I guess.
|
|
#
?
Oct 27, 2011 00:48
|
|
|
If you don't come here a lot and are looking for things to do/eat/see let me know. FAB2 is getting a good look from me right now, but honestly we are barely scratching the surface of what our 7k's can do today. I've budgeted and included them in my 2012 budget/roadmap, but unless the server/storage teams really step up their requirements I doubt I'll be making that move.
|
|
#
?
Oct 27, 2011 03:01
|
|
|
Does anybody know of a decent product for managing ACLs? At the moment I'm using baseline configurations in Ciscoworks to keep ACLs consistent across our network devices but it's pretty cludgy. I know there used to be an ACL manager product for ciscoworks but its long been end of sale.
|
|
#
?
Oct 27, 2011 03:43
|
|
|
|
| # ? May 18, 2024 03:21 |
|
|
ruro posted:Does anybody know of a decent product for managing ACLs? At the moment I'm using baseline configurations in Ciscoworks to keep ACLs consistent across our network devices but it's pretty cludgy. I know there used to be an ACL manager product for ciscoworks but its long been end of sale. Cisco security manager!
|
|
#
?
Oct 27, 2011 05:25
|
|