Protein toxins such as Ricin and Pseudomonas exotoxin (PE) pose major public health challenges. Both toxins depend on host cell machinery for internalization, retrograde trafficking from endosomes to the ER, and translocation to cytosol. Although both toxins follow a similar intracellular route, it is unknown how much they rely on the same genes. Here we conducted two genome-wide RNAi screens identifying genes required for intoxication and demonstrating that requirements are strikingly different between PE and Ricin, with only 13% overlap. Yet factors required by both toxins are present from the endosomes to the ER, and, at the morphological level, the toxins colocalize in multiple structures. Interestingly, Ricin, but not PE, depends on Golgi complex integrity and colocalizes significantly with a medial Golgi marker. Our data are consistent with two intertwined pathways converging and diverging at multiple points and reveal the complexity of retrograde membrane trafficking in mammalian cells.