Figure 8.1 from the datasheet shows the timing for data clocked from the MCU to the TM1638, which the TM1638 samples on the rising edge. The clock for SPI devices, which shiftOut is intended to control, is low when idle, whereas the TM1638 clock is pulled high when idle with a 10K pull-up. For data that is sent from the TM1638 to the MCU, data is supposed to be sampled after the falling edge of the clock. After doing some testing at high speed (>16Mhz) that revealed problems reading the buttons pressed, I decided to rewrite the library using open-drain communication, and trying to adhere to the TM1638 specs.
With open-drain communications, the MCU pin is switched to output low (0V) to signal a zero, and switched to input to signal a 1. When the pin is in input mode, the pull-up resistor on the line brings it high. The Arduino code to do this is the pinMode function, using OUTPUT to signal low, and INPUT to signal high. The datasheet specifies a maximum clock rate of 1Mhz, but that appears to be more related to the signal rise time with a 10K pullup than a limit of the speed of it's internal shift registers. For example I measured a rise time from 0V to 3V of 500ns with my scope:
To account for possibly slower rise times when using longer and therefore higher capacitance wires, before attempting to bring the clock line low, I read it first to ensure it is high. If is not, the code waits until the line reads high. As an aside, you may notice from the scope shot, the lines are rather noisy. I experimented with changing the decoupling capacitors on the board, but it had little effect on the noise. I also noticed a lot of ringing due to undershoot when the clock, data, or strobe lines are brought low:
I found that adding a series resistor of between 100 and 150 Ohms eliminates the ringing with minimal impact on the fall time.
The updated TM1638 library can be found in my nerdralph github repo: