The song that Artie composed for his dad (entitled "Nocturne for Artie's Dad") was played and composed by the actor who plays Artie, Saul Rubinek.
As "Henry Deacon" on the series Eureka (2006), Joe Morton plays a figure people look to for help and guidance playing a more morals-centric figure as the man we first see in this episode (interestingly, two of his costars from that show guest-starred in the previous episode, also as different characters--in spite of the fact that Warehouse 13 and Eureka are shown to exist in the same universe). He also went on to play a reverend ("Arthur Finer") in God Friended Me (2018).
At approximately 26 minutes in, when Pete and Myka are discussing the history of the prison, among the photographs of miners on the wall behind Pete is a photograph of the hanging of the Lincoln conspirators.
Saul Rubinek (Artie Nielson) and Steve Cumyn (Pete's Dad) both appeared in the series A Nero Wolfe Mystery episode Prisoner's Base.