Inflammation of the rectum of dogs can be due to rectal polyps, rectal prolapse or anal sac disease.
In rectal prolapse, rectal tissues protrudes through anus. Exact cause of this condition is not known but it is considered that gastrointestinal parasites can contribute in the development of this disorder. Rectal prolapse requires immediate treatment.
Another possibility is anal glands disease. These anal glands are expressed during defecation. Some conditions can cause swelling in these glands like infection of anal glands or abscess formation, impaction, and tumors of anal glands. Diagnosis is by physical and rectal examinations and blood tests. Antibiotics are used to treat infections and if no infection then only expressing the anal glands will solve the problem. Tumors are treated by removal of affected anal gland.
There can be few reasons of anal swelling in dogs. Most commonly, it can be anal sac disease. Anal glands contain some sort of secretions that are released at the time of passing stools. Sometimes, these secretions become highly viscous and unable to release. This condition can leads to impaction. Sometimes, tumors and infection can develop in these anal glands. All these conditions can lead to swelling of of anus. If you know how to express anal glands then do it know, otherwise take him to vet. If there is infection then use of antibiotics can be helpful. Anal gland tumors are removed by surgery.
Other possibility of anal swelling in your dog is anal or rectal prolapse. Etiology of this condition is not known but intestinal parasites have some role. This condition is also treated by vet. So, it will be good to get opinion from your local vet.
I have friends who have a dog who has his glands drained regularly. As well as the vet, your groomer should be able to drain them.