|
which one? the name game, or ospf hard? or both? i am not a genius but i can make a totally stubby area for a group of servers to talk to each other work. the name game specifically is dumb because you are baking network topology into app config or hijacking hostnames on specific machines to force the routing you want.
|
#
?
Feb 12, 2024 19:08
|
|
|
|
| # ? Jun 11, 2024 16:17 |
|
|
the name part what in gods name are you guys doing why is “link aggregation” being thrown around?
|
|
#
?
Feb 12, 2024 19:12
|
|
|
theyre trying to take advantage of a fast internal magic rdma memory to memory hypervisor network when two guests are on the same host instead of having the hypervisor virtual switch handle it. its a latency/cpu cost thing. the usecase is real the existent solution is dumb.
|
|
#
?
Feb 12, 2024 19:17
|
|
|
if you're routing your rdma packets, you already lost the war rocev2 is loving stupid, ib4ever
|
|
#
?
Feb 12, 2024 19:40
|
|
|
okay i'll bite as someone doing a lot of vms on hosts all over the place (libvirt w/ kvm/qemu) should i be doing something more clever than bridging these to the host's main interface (yes it's a flat network but the number of nodes at any given time is <100)
|
|
#
?
Feb 12, 2024 20:05
|
|
|
yea see but when you can use an ip address that gives you rdma to the target, or a different ip address in a different subnet because its a different L2 domain for vanilla ethernet - how do you know to choose the rdma capable non routed link? eg: every virtual machine has a real ethernet link to the external network and the world, and also a magic internal rdma link on a private subnet to talk to peers in the same hypervisor. how do you tell an application to use one link to talk to peers in the hypervisor, and a different link to talk to the world? stupid dns tricks and /etc/host fuckery? bridge the private rdma link to the world and use real ips everywhere? bake the network topology into the app config? what happens when you want to relocate a guest to a different hypervisor and the neighbors/peers/remote endpoints all switch around? ospf specifically handles all the above, but nooooooo its gross and weird and everyone wants to solve a routing question using some other method. as far as i can tell there are no good answers, just different kinds of bad
|
|
#
?
Feb 12, 2024 20:16
|
|
|
psiox posted:okay i'll bite as someone doing a lot of vms on hosts all over the place (libvirt w/ kvm/qemu) depends on what youre doing in those vms. 90% of the time bridges to the real adapter are the right answer. but if you have a group of vms that stream gigs of data to each other constantly, on the same host it can be a major latency/throughput/cpu time savings to have a better local network in the hypervisor for them to use.
|
|
#
?
Feb 12, 2024 20:21
|
|
|
the ospf wikipedia article is a real gem Open Shortest Path First (OSPF) is a routing protocol for Internet Protocol (IP) networks. It uses a link state routing (LSR) algorithm and falls into the group of interior gateway protocols (IGPs), operating within a single autonomous system (AS). OSPF gathers link state information from available routers and constructs a topology map of the network. The topology is presented as a routing table to the internet layer for routing packets by their destination IP address. ... OSPF is an interior gateway protocol (IGP) for routing Internet Protocol (IP) packets within a single routing domain, such as an autonomous system. It gathers link state information from available routers and constructs a topology map of the network. The topology is presented as a routing table to the internet layer which routes packets based solely on their destination IP address.
|
|
#
?
Feb 12, 2024 20:28
|
|
|
Redistribution into an NSSA area creates a special type of LSA known as type 7, which can exist only in an NSSA area. An NSSA ASBR generates this LSA, and an NSSA ABR router translates it into a type 5 LSA, which gets propagated into the OSPF domain. A newly acquired subsidiary is one example of where it might be suitable for an area to be simultaneously not-so-stubby and totally stubby if the practical place to put an ASBR is on the edge of a totally stubby area. In such a case, the ASBR does send externals into the totally stubby area, and they are available to OSPF speakers within that area. In Cisco's implementation,
|
|
#
?
Feb 12, 2024 20:31
|
|
|
yea if youre going to use ospf to interconnect a hodgepodge of routers on a campus with a “whatever was cheapest at the time” set of uplinks between buildings it will totally do that youre gonna want to murder whoever named all the terms though.
|
|
#
?
Feb 12, 2024 20:43
|
|
|
it is cool as poo poo to watch the routing tables all converge on new pathing when you take a shared link down though a ping flood to an external device only sees 2-3 dropped packets on a ssh session that was running over the link you killed.
|
|
#
?
Feb 12, 2024 20:47
|
|
|
fresh_cheese posted:yea see but when you can use an ip address that gives you rdma to the target, or a different ip address in a different subnet because its a different L2 domain for vanilla ethernet - how do you know to choose the rdma capable non routed link? the route table on each host would have a direct route for the non routed link which would be preferred for that specific subnet, and all other traffic would take the routed link, unless im missing something? that should just happen by default
|
|
#
?
Feb 12, 2024 21:05
|
|
|
node1: routed ip: 10.20.30.10 rdma not routed ip: 192.168.0.10 routes: 10.20.30.0/24 direct 192.168.0.0/16 direct default/0.0.0.0 via 10.20.30.1 node2: routed ip: 10.20.30.11 rdma not routed ip: 192.168.0.11 routes: 10.20.30.0/24 direct 192.168.0.0/16 direct default/0.0.0.0 via 10.20.30.1 are you sayin i should add these routes to the above? node1: 10.20.30.11/32 via 192.168.0.11 node2: 10.20.30.10/32 via 192.168.0.10 and automate the management thereof as i place virtual machines on hypervisors and them move them around? congrats you just reinvented ospf more or less.
|
|
#
?
Feb 12, 2024 21:30
|
|
|
fresh_cheese posted:node1: I don’t understand why the problem you’re presenting exists but ok so is the problem you’re trying to solve is when a host wants to communicate with another host via its routed address, rather than communicate with the host via its non-routed address, you need something to force that traffic to take the non-routed path while preserving the network addressing of the routed path? what is causing this situation to exist? why do the hosts to try to communicate with one another via their routed addresses and not their non-routed addresses? is this just a dns issue where you just need to do split horizon?
|
|
#
?
Feb 12, 2024 22:00
|
|
|
the most basic basic bit of info is that there are two different networks. one is very very fast, but only gets you to neighboring virtual machines in the hypervisor. thats 192.168.0.0 the other is the network that talks to the rest of the world. thats 10.20.30.0 re your question: yes. how do you let application code get the benefit of the fast network to adjacent peers while maintaining connectivity to remote peers that are unreachable over the fast network? and also do so across an environment running dozens/hundreds of hypervisors that contain these little island networks? and do so in a way such that you can start up a virtual machine on any one of those dozens of hosts and theyll just figure it out to get to the local peers with the fast network while still having access to everything else via the real network. without going so far as ospf - split horizon dns was the next least bad option. a dns service on the fast private network responding to all requests for locally fast attached hosts with the 192.168 fast ip, and forwarding all other requests to external dns to supply 10.x normal IPs. its a colossal pita to automate the maintenance of those per hypervisor zones though as you move virtual machines around.
|
|
#
?
Feb 12, 2024 22:22
|
|
|
My comedy answer is consul
|
|
#
?
Feb 12, 2024 22:27
|
|
|
tortilla_chip posted:My comedy answer is consul thats just hurtful. im trying to save 0.4 ms of rtt and 1/32 of a cpu worth of compute time and youre just making GBS threads kubernetes everywhere. geddafuqouddahere
|
|
#
?
Feb 12, 2024 22:36
|
|
|
fresh_cheese posted:the most basic basic bit of info is that there are two different networks. thank you for the more detailed explanation, now i understand the problem better i wonder if doing some sort of mDNS fuckery on the non-routed network of each hypervisor could allow a lot to be self-configured
|
|
#
?
Feb 12, 2024 22:43
|
|
|
if your rdma network sits on top of ip or even ethernet, you are already hosed
|
|
#
?
Feb 12, 2024 22:48
|
|
|
Asymmetric POSTer posted:what is causing this situation to exist? why do the hosts to try to communicate with one another via their routed addresses and not their non-routed addresses i guess i dont understand the question? theres two paths between hosts that are on the same hypervisor. each path is via a unique subnet. the subnets are not routed to each other. ( why are the little fast subnets not routed to the world which would indeed make the entire problem go away you ask? because routing packets costs host cpu cycles that are better spent committing db transactions that make the real money and the boxes run at high 90% utilization so every cycle matters. )
|
|
#
?
Feb 12, 2024 22:49
|
|
|
im thinkin dhcp and dynamic split horizon dns on a network services machine on each hypervisor might do the needful. network people get real grumpy about letting network services run on hosts though. that may take some beers to design a solution at an offsite with the network team. and “if youre doing rdma over ethernet you already failed”: whaddyo mean remote memory access over a lossy transport doesnt sound exciting to you?
|
|
#
?
Feb 12, 2024 23:00
|
|
|
I knew all that bullshit about stun areas 10 years ago like the back of my hand and I've never used it even once
|
|
#
?
Feb 13, 2024 09:17
|
|
|
fresh_cheese posted:theyre trying to take advantage of a fast internal magic rdma memory to memory hypervisor network when two guests are on the same host instead of having the hypervisor virtual switch handle it. how much extra money does it cost to just not use the rdma thing ever and not give a poo poo
|
|
#
?
Feb 14, 2024 08:42
|
|
|
im trying to imagine a real usecase for opportunistic rdma and not coming up with anything. either it matters and you're willing to take the two minutes to place VMs where they're guaranteed to have that fast link, or it doesn't matter so gently caress it. caring but only just enough to make a lot of extra work and add a lot of complexity to your infrastructure is pretty close to peak clever boy without enough real work to do tbqh
|
|
#
?
Feb 14, 2024 08:44
|
|
|
yeah this is feeling real "solution in search of a problem" except the solution is one of the worst ones anyone could come up with
|
|
#
?
Feb 14, 2024 08:54
|
|
|
its for financial transactions when youre running millions/hr with a tight sla. yea for flappy bird fart app it wouldnt be worth it
|
|
#
?
Feb 14, 2024 13:02
|
|
|
High frequency trading bloodsucking company spotted
|
|
#
?
Feb 14, 2024 13:53
|
|
|
fresh_cheese posted:its for financial transactions when youre running millions/hr with a tight sla. i knew it
|
|
#
?
Feb 14, 2024 16:02
|
|
|
i thought that was obvious from the problem domain
|
|
#
?
Feb 14, 2024 16:36
|
|
|
sounds like it's already too slow since you're using an operating system in a vm, reimplement everything in a bespoke asic connected to the fibre module
|
|
#
?
Feb 14, 2024 17:11
|
|
|
fresh_cheese posted:its for financial transactions when youre running millions/hr with a tight sla. why don’t you have a rdma capable network then why is your vm network switch stupid besides, bgp on the host is the current hotness if you are fully committing
|
|
#
?
Feb 14, 2024 17:33
|
|
|
fresh_cheese posted:its for financial transactions when youre running millions/hr with a tight sla. you didnt answer my question just waved your hands at a pile of money. how much smaller does the pile get if you dont do this dipshit opportunistic fuckery
|
|
#
?
Feb 14, 2024 17:46
|
|
|
i fully understand the masculine urge to believe that what you are doing is cool and interesting, believe me i do, but you need to overcome that and think rationally about what you are doing and what purpose it serves
|
|
#
?
Feb 14, 2024 17:47
|
|
|
like jesus christ "tight sla" well ok fine but also you may or may not actually have the magic hookup that makes the latency good and you need to be ready to handle either case? love 2 make my sla compliance nondeterministic
|
|
#
?
Feb 14, 2024 17:48
|
|
|
bare metal plus infiniband. raw verbs or ucx, no ip
|
|
#
?
Feb 14, 2024 17:50
|
|
|
Somebody fucked around with this message at 05:27 on Feb 15, 2024 |
|
#
?
Feb 15, 2024 04:10
|
|
|
text me Somebody fucked around with this message at 05:28 on Feb 15, 2024 |
|
#
?
Feb 15, 2024 05:26
|
|
|
hft? gently caress no this is generic moving money around bank poo poo.
|
|
#
?
Feb 15, 2024 13:08
|
|
|
gently caress, man thats just hurtful
|
|
#
?
Feb 15, 2024 13:30
|
|
|
|
| # ? Jun 11, 2024 16:17 |
|
|
so some sort of normal frequency trades we could call them nfts
|
|
#
?
Feb 15, 2024 14:11
|
|
