The effect of inelastic scattering on quantum electron transport through layered semi-conductor structures is studied numerically using the approach based on the non-equilibrium Green's function formalism of Keldysh, Kadanoff, and Baym. The Markov assumption is not made, and the energy coordinate is retained. The electron-photon interaction is treated in the self-consistent first Born approximation (SCFBA). The Pauli-exclusion principle is taken into account exactly within the SFCBA. The retention of the energy coordinate allows the calculation of a number of quantities which give insight into the effect of inelastic scattering on electron transport: the effect of inelastic scattering on the occupation of the energy levels, the density of states, the energy distribution of the current density, and the power density is calculated from a quantum kinetic equation for actual device structures under high bias. The approach is used to study the effect of emitter quasi-bound states on the I-V characteristic of resonant tunneling diodes (RTD's) the effect of barrier asymmetry on the phonon-peak in RTD's, and energy balance and heat exchange in mescopic systems.