but I've noticed a lot of those people don't really put in the effort needed on their end.

A matter of priorities. Congrats on doing what you needed to do (despite the hardships) keeping your focus on those kids. They will remember and the bonds will be strong as they grow older. While not the way one wants it to unfold, you are teaching them important lessons in love that many are unfortunate never to understand.