In recent years, Edge computing has emerged as a new paradigm that can reduce communication delays over the Internet by moving computation power from far-end cloud servers to be closer to data sources. It is natural to shift the design of cloud-based IoT applications to Edge-based ones. Activity recognition in smart homes is one of the IoT applications that can benefit significantly from such a shift. In this work, we propose an Edge-based solution for addressing the activity recognition problem in smart homes from multiple perspectives, including architecture, algorithm design and system implementation. First, the Edge computing architecture is introduced and several critical management tasks are also investigated. Second, a realization of the Edge computing system is presented by using open source software and low-cost hardware. The consistency and scalability of running jobs on Edge devices are also addressed in our approach. Last, we propose a convolutional neural network model to perform activity recognition tasks on Edge devices. Preliminary experiments are conducted to compare our model with existing machine learning methods, and the results demonstrate that the performance of our model is promising.